I would say making the model larger and larger will soon reach the "the point of diminishing returns".
Even if the model might still get 'smarter' this might not be a good thing because the consumer (us) might not understand at all what the ai is doing. Or it's intelligence will just not be of practical value.
After this it will be hardware optimizations and tooling specializations (having different models for different tasks) and running the whole model might not be too expensive anymore.
Will they be outcompeted by cheaper Chinese competitors or open models? Possibly.
I hope we reach the point I can run the whole darn thing on a old laptop.
After this it will be hardware optimizations and tooling specializations (having different models for different tasks) and running the whole model might not be too expensive anymore. Will they be outcompeted by cheaper Chinese competitors or open models? Possibly.
I hope we reach the point I can run the whole darn thing on a old laptop.