I'm trying and failing to find value running a potential Qwen 3.8 27b dense model on a 16 core, 128 GB of ram, 2080ti box. Yes, the GPU yells for help, but the problem is that no math works to upgrade this machine even when pouring $200 in rent every month into the large model providers...
How are you all justifying economical use of these local models right now? What's the cost efficient way to do this and do better (even with models evolving over time and losing now vs later) than the big labs?
How are you all justifying economical use of these local models right now? What's the cost efficient way to do this and do better (even with models evolving over time and losing now vs later) than the big labs?