Comment by turkeygizzard

Thank you for articulating this. I remember similar problems and arguments arising after RNNs and CNNs became massively successful. People argued that training larger models would be infeasible for several reasons that all were made moot by Attention Is All You Need. Somebody seems to always figure out a new approach