blagui

The real gains are far less than the advertised. Tests ton TFTOPS show 25% gains at 200Watt cap. While mostly prefil benefit from that less decoding which make the real gains 7-8%. Using at full power:

This is a benchmark test with Qwen 27B Int4 with 4x3090 and using full batching on VLLM to squeeze every bit of the GPU's.

Error rate is crazy high. And comparing to low models and omit a lot of other bigger open models.

7 t/s not sure how you can use this? And if it start thinking a bit. It works yes but that's really very slow. On top it's very low quant that impact the model quality.

You have 4 thinking levels.

You can disable it. It's well known issue in Qwen, previous releases I would disable it by default.

Also xhigh seem a new thing.

A 9B beating frontiers" No! But nailing specialized tasks Yes.

GLM is the model that will sink the frontier labs.

Recall last year deepseek? And 18 month's later? What changed?

Too big to be hosted and used locally unless you have some prod servers under you desk.

And those aiming to fit with Q2 or Q1. It's not even worth it to destroy the models to claim it's still alive after cutting all the limbs.

So the sweet spot for dev in 2026 is 64k context windows? Are we back in 2024?

As more context will degrade a lot the t/s. On top this is 1 slot.

If you use sub agents the kv cache will be invalidated with colliding request and make it even slower.

So the in real world 256k (the max qwen offer) and using 3-4 slots the numbers are very different.

This is the major issue with so many postes over local models not benchmarking real world use. Real context and not taking this in context.

If you use 1 slot the issue, you loose the ability of using sub agents when exploring and all end up in the main agent context overloading it, triggering compactation and oh boy with 64k context that compecation will be an endless loop.

What tasks you would really be able to do with 64k context 1 agent? For sure so quick edits but not complex planning where you need to ingest a lot files and end up loosing 80% of the ingested files to compactation.

How you can do dev in 2026 using 64k context and without sub agents?

The benchmark seemed fine until I saw that.

If you use sub agents, they will overwrite the cache and each request will trigger full reprocessing. Have fun with that as it will crash the t/s metrics on each prefill on top of the max 64k including input + output is a major blocker.

If you push the context higher and add parallel slots the requirements will be far higher and the numbers less shiny.