I find these "combination benchmarks" entirely useless. It promotes benchmaxing on "knowledge". When the most important thing typically is ability. You almost never have tasks that span such wide domain.
For example on programming, terminal tasks, troubleshooting etc Qwen3.8-Flash-Next is almost as good as opus 4.8. On "knowing what it doesn't know" it is far behind opus, but leads all other OS models I tested by a wide margin.
It out classes Glm5.3 flash and leaves models like DeepSeek 4.1 far in the dust. But you still get those silly aggregate benchmarks where GLM and DeepSeek are higher.
It is a small model, only 120B anyone that expects it to excel on general knowledge or to have a bajillion ready made workflows memorised is kidding themselves.
Everyone knows the bigger the model the better it will do on raw content of knowledge. But you know what compensates for lack of it? A good harness. But there is no compensating for lack of ability.
So in short I find all those aggregate benchmarks useless.
For example on programming, terminal tasks, troubleshooting etc Qwen3.8-Flash-Next is almost as good as opus 4.8. On "knowing what it doesn't know" it is far behind opus, but leads all other OS models I tested by a wide margin.
It out classes Glm5.3 flash and leaves models like DeepSeek 4.1 far in the dust. But you still get those silly aggregate benchmarks where GLM and DeepSeek are higher.
It is a small model, only 120B anyone that expects it to excel on general knowledge or to have a bajillion ready made workflows memorised is kidding themselves.
Everyone knows the bigger the model the better it will do on raw content of knowledge. But you know what compensates for lack of it? A good harness. But there is no compensating for lack of ability.
So in short I find all those aggregate benchmarks useless.