Local models cannot win. Local GPUs for gaming win over nvidia geforce now because of network bandwidth and latency and that cost is within the same order of magnitude as owning a GPU (though still substantially cheaper)
With LLMs, network latency hardly matters compared to overall response generation time, and GPUs processing multiple requests simultaneously have total tok/s throughout much higher than when handling a single request (unlike game streaming, where the main cost benefit of putting GPUs in the cloud is just keeping them active 24/7). Cloud inference is/will be multiple orders of magnitude cheaper than owning a GPU for any reasonable amount of usage over the life of the GPU.
With LLMs, network latency hardly matters compared to overall response generation time, and GPUs processing multiple requests simultaneously have total tok/s throughout much higher than when handling a single request (unlike game streaming, where the main cost benefit of putting GPUs in the cloud is just keeping them active 24/7). Cloud inference is/will be multiple orders of magnitude cheaper than owning a GPU for any reasonable amount of usage over the life of the GPU.