It says 46 tok/sec for Qwen3.6-27b 4 bit on RTX5090, but I get 60-70 tok/sec on a small context like that (5 bit weights with 8 bit KV cache), although it does slow down to 40 tok/sec on large contexts. If I add ngram speculation, then it can increase to 100+ tok/sec on code.
There are many knobs to tweak to make it faster, which the site ignores.
There are many knobs to tweak to make it faster, which the site ignores.