sharmajai

Yes, that's what I meant, thanks for clarifying!

If you want to draw from history there are multiple successful examples of this in science and engineering at least: Bletchley Park, the Manhattan Project, Moon Landings.

A cloud model can also get neutered anytime and/or start using more tokens while charging you the same or more per token.

I see, maybe your use case is indeed pathological for Qwen 3.8 27b. Another thing to try if you are using llama.cpp is "reasoning budget". That makes the thinking stop after the budget has been reached and inserts a custom message you can choose, so something like "you have thought for too long, now continue with the execution ..."

For what it's worth, if you haven't already, you can also let it run overnight (if you have compaction enabled) to see if it ever gets out of that hole. The reason `xhigh` is the default is because 3.8 is trying to optimize for long horizon tasks where monitoring its every single thinking misstep might not be a good use of our time.

Without knowing anything about your benchmark, it might be your harness at fault here. I say this because with longer thinking you run the risk of filling up your context faster and you need a good context compaction strategy in your harness to mitigate that.

I have heard good things about Pi which supports auto-compaction, but I can't personally vouch for it since I use my own.

This confirms a theory I have to explain the minimal loss in quality when using lower quants (I use IQ3_XXS with an 8-bit KV cache) and the XHIGH (default) thinking level.

It's well-known that while quantization affects the sampling probability distribution (given the same context, which next token is the most probable), Qwen 3.8 27b seems to offset that by just thinking more and as a result eventually finishing the task (benchmark or otherwise).

So as long as the thinking (albeit longer) is sound, this leads to the same success rate (as shown in the article) but potentially at the cost of more tokens and hence more time.

I think it'll be further useful to chart each quantization's used tokens as well, in addition to the success rate.

Thanks for doing and sharing the research!

Really feels like AGIPO is here.

Thanks for the movie suggestion, looks very interesting, will check it out. Also, the Pixar movie Soul has a similar message.

You'd be a great CEO, preaching people to tie their worth to their output in life.

Who said investing is _only_ for "capital and capital appreciation"? It can also be for social good.

They have a very opaque and subjective permanent residency program. So while they get all the benefits out of you as an immigrant, they may provide none in return.