Yes, but if you can run a sufficient LLM on a $2,000 laptop, then the cost to serve it from the cloud will be similarly cheap. (e.g. reserve an appropriately sized EC2 instance for pennies on the dollar)
It's a highly competitive market. Companies aren't going to pay 100k/year to run a model on something that can run on a 2k consumer grade device.
128GB of gpu accessible/fast RAM can be had for $5000 on a macbook pro today. What will it be 3-4 years from now on linux/windows machines?
And we still haven't seen any SoC providers try to optimize for RAM capacity over compute yet.
It's a highly competitive market. Companies aren't going to pay 100k/year to run a model on something that can run on a 2k consumer grade device.
128GB of gpu accessible/fast RAM can be had for $5000 on a macbook pro today. What will it be 3-4 years from now on linux/windows machines?
And we still haven't seen any SoC providers try to optimize for RAM capacity over compute yet.