Maybe “TF 3” or whatever they call it will be ergonomic and a pleasure to use, but that was the promise of TF 2, and unless you wanted to use Keras it was anything but. I’m glad I have been able to work only in PyTorch and Jax. Maybe TF will get better but I’m not holding my breath. On the other hand, XLA is very nice, and I hope they continue to develop it.
I hope torch has a convincing distributed tensor API coming soon. Their development on ShardedTensor seems to have slowed or stopped recently, so TF’s DTensor is definitely ahead, which is a shame. And of course TF’s ecosystem for the whole lifecycle is more mature with TFX, TF.js, etc, but torch is slowly closing those gaps, and hopefully that will continue.
I hope torch has a convincing distributed tensor API coming soon. Their development on ShardedTensor seems to have slowed or stopped recently, so TF’s DTensor is definitely ahead, which is a shame. And of course TF’s ecosystem for the whole lifecycle is more mature with TFX, TF.js, etc, but torch is slowly closing those gaps, and hopefully that will continue.