If model performance continues to scale with model size, I have a hard time seeing how local models will have any chance of competing with models hosted on datacenter hardware.
1. There are strong economies of scale in hosting inference (batched prompts, high uptime, shared infrastructure).
2. There are physical limits on how much memory we will be able to produce over the next few years. Demand will probably scale at least as fast as production does, so we won't be saved by falling prices.
1. There are strong economies of scale in hosting inference (batched prompts, high uptime, shared infrastructure).
2. There are physical limits on how much memory we will be able to produce over the next few years. Demand will probably scale at least as fast as production does, so we won't be saved by falling prices.