The current crop of LLMs is like Paleozoic megafauna, or like Egyptian Pyramids. It takes a few relatively simple approaches and stretches them wildly, using colossal computing resources.
Live systems in nature seem to solve similar problems with way less compute available. There should be better architectures.
Also, as somebody said, every exponential growth curve is a lower part of a sigmoid. LLMs will plateau at some level. That level may be impressively high though.
Live systems in nature seem to solve similar problems with way less compute available. There should be better architectures.
Also, as somebody said, every exponential growth curve is a lower part of a sigmoid. LLMs will plateau at some level. That level may be impressively high though.