Comment by tourist_on_road

Transformers gained popularity due to the scalable nature of the architecture and how well it can be parallelized on existing GPU/XLA hardware. Modeling is always conditioned on the hardware available at hand. Transfomers lack inductive bias which make it generic building blocks unlike CNN/RNN like models and by injecting inductive bias like positional encoding, it can be well translated to various domains.