Comment by the8472

Yeah that's pretty much what gwern argues here[0]. Or to adapt another proverb: to predict the next token you first need to model the universe.

[0] https://gwern.net/scaling-hypothesis#gwern-difference--effic...