Comment by anigbrowl

The hardware is certainly not cheap, but you can put 512gb of vram on your desk if you are willing to throw $30k at Apple, which is a manageable expense for a lot of small businesses. Bridging a few devices is one way to run large models, r ablation and quantization can do a lot too. You definitely need a data center if you aim to make the model directly accessible to the public, with whatever fine tuning or secret sauce you add. If you want to run it in-house as a coding engine or research tool you can likely reduce its footprint substantially.