this is not only not true , it has no basis's in reality.
In the "real" world there are tradeoff's and constraints.
scaling does not work infinitely , and products which are delivered by good engneering cultures have a non linear growth ( bad vs very good ).
comming back to research one of the frontier model's deepseek was able to come close to SOTA with a relatively small budget because of one of their mixture of experts approach.
comming back to research one of the frontier model's deepseek was able to come close to SOTA with a relatively small budget because of one of their mixture of experts approach.