Qwen3.8-27B is surprisingly effective, somewhere in the vicinity of Opus 4.6, but it needs to do a lot of inference-time computation. i.e. anything moderately involved requires xhigh reasoning and it will spend literally tens of thousands of tokens turning over a problem before emitting the answer. The answer/code/whatever is often surprisingly good, but if you’re in the low tens of tok/s you’ll be waiting a long time to get it.
As sibling said, you won’t hit the same quality using a 2 bit quant like this project, which is unfortunate but that’s the reality today.
As sibling said, you won’t hit the same quality using a 2 bit quant like this project, which is unfortunate but that’s the reality today.