I don't think your conclusion of "hitting the wall on intelligence" is warranted.
It makes more sense to believe that scaling has hit the wall on available text data to train on, and that to continue scaling, along with whatever emergent properties arise they need much more data than exists as text.
There are orders of magnitude more data as video, audio, and images and this is what they intend to use to continue scaling.
It makes more sense to believe that scaling has hit the wall on available text data to train on, and that to continue scaling, along with whatever emergent properties arise they need much more data than exists as text.
There are orders of magnitude more data as video, audio, and images and this is what they intend to use to continue scaling.