On my RTX 5090, Qwen3.6-35B-A3B is my go to for coding, but Gemma-4-26b-a4b (Q4_0) is blazing fast and super high quality. I sometimes use the dense cousins of these models (Qwen3.6-27B QB_0 and Gemma4-31B-QAT Q4_0) but they can be very slow. Integrating web search and code calling via Open-WebUI makes it so that I usually don't need to reach for Claude.
Here are the tok/s I get:
- Gemma-4-26B-A4B (Q4_0) = 214 tok/s
- Gemma4-31B-QAT (Q4_0) = 58 tok/s
- Qwen3.6-35B-A3B (QB_0) = 30 tok/s
- Qwen3.6-27B (QB_0) = 9 tok/s
EDIT: Updated tok/s after updating llama.cpp