Occam's razor tells us that if it's a great architecture/technical breakthrough, it would have taken the world by storm by now. Similar to the original transformer paper and model. Since 2017, the only successful models are variations of transformers. RNNs are no where in the picture.
Simply believing a architecture is superior doesn't make it so. Nothing converges and performs as good as a model with attention in both training and inference. The difference is night and day.
Simply believing a architecture is superior doesn't make it so. Nothing converges and performs as good as a model with attention in both training and inference. The difference is night and day.