Running Qwen 3.8 27B on a 5090, 8bit quant and 160k context fills the card but I get about 200 tps. Not as good as claude but its the first local model I have tried where it could easily replace claude. The local setup takes about twice as long as claude to complete a task to the same output quality. Electricity is expensive here so running a 600w gpu 24/7 ends up costing about the same as just buying credits. Free heating for winter and no spying are still worth it.
I went for the 5090 for the software support but I have so many issues running cuda on fedora. Kinda wish I had bought a Nvidia spark or something that was an AI only machine.
I went for the 5090 for the software support but I have so many issues running cuda on fedora. Kinda wish I had bought a Nvidia spark or something that was an AI only machine.