VHRanger
Joined 9 years ago
Joined 9 years ago
Thanks for the recap
For gemma4 26B, same quantization, I get >200TPS.
Also note that qwen is extremely inefficient in reasoning; the reasoning chains are ~3x longer than gemma on average
The best LLM benchmarks test around the margins of those behaviors, tasks that are difficult and correlate with usefulness while being removed enough to stay unpolluted
The local inference space is pretty good nowadays.
Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.
It's better to have a smaller core of highly engaged people than a mass of disengaged eyeballs glazing over.
I remember
It's just sparkling memory safe high performance software
Canada has similar levels of economic development.
It's really not that complicated: controlling for general crime levels, guns drastically increase murder rates.
Cosmic works great for a laptop. But it's a PITA for a desktop. It doesn't deal with multi monitor setups well. There's a recent new bug where the system hardlocks on monitor power state changes, which is unacceptable.
So: great for single screen laptop, not good for desktop or server