It's going to boil down to cost of the hardware vs. pay per use cloud models. I'm on the 100$/month chatgpt plan (~88euro). With all the resets, credits, and all the rest I typically don't run out of tokens. This is fine for me. I might go to the 200$/month plan at some point.
In comparison, my highend mac book pro costs 105 euro per month (leasing cost). That's not quite enough to run the bigger models and besides running the GPU on a laptop constantly makes it a bit hostile for my lap. And the model I would need a lot more memory for the bigger models. It's nice for experimenting though.
If you get your own GPUs in some kind of server form, you could run those locally or in a data center. Or you just get hardware resources from your favorite cloud provider. Hetzner (one of the more affordable ones) GPU servers start at around 200 euro per month. I know some people that use that with open weight models. Works fine, especially if you can share the resources between multiple users.
At which point of course you are doing the same as cloud based providers are doing. But probably with a bit worse economics. It all boils down to whether you are actually saving money or paying for a lot of underutilized hardware that's running 24x7 when your people actually only show up about 8x5.
I don't think most companies should be doing that. It sounds like they'd end up with a lot of idling over capacity at a premium. The only valid reasons to do this would be securing access to GPUs or some very urgent need to keep things on premise (which a lot of companies insist is true for them until they realize how much they are going to pay).
In comparison, my highend mac book pro costs 105 euro per month (leasing cost). That's not quite enough to run the bigger models and besides running the GPU on a laptop constantly makes it a bit hostile for my lap. And the model I would need a lot more memory for the bigger models. It's nice for experimenting though.
If you get your own GPUs in some kind of server form, you could run those locally or in a data center. Or you just get hardware resources from your favorite cloud provider. Hetzner (one of the more affordable ones) GPU servers start at around 200 euro per month. I know some people that use that with open weight models. Works fine, especially if you can share the resources between multiple users.
At which point of course you are doing the same as cloud based providers are doing. But probably with a bit worse economics. It all boils down to whether you are actually saving money or paying for a lot of underutilized hardware that's running 24x7 when your people actually only show up about 8x5.
I don't think most companies should be doing that. It sounds like they'd end up with a lot of idling over capacity at a premium. The only valid reasons to do this would be securing access to GPUs or some very urgent need to keep things on premise (which a lot of companies insist is true for them until they realize how much they are going to pay).