hgoel

Ahh okay, that makes more sense! Sorry for the snark.

Stand out from what? Are you seriously arguing that the only things keeping Signal from being banned are that it somehow camouflages itself?

I've been torn on Proton for a while, but looks like I won't be renewing my subscription...

It seems he's responding to a question asking if he thinks people will be able to run local AI on their own computers. Not arguing on higher prices. So it comes off more as "people shouldn't get to have enough RAM to run local AI because it's an inefficient use of resources" rather than the capitalist "I can afford to pay more for RAM so I should get priority".

The other way to read this is that the US frontier companies don't care about public benchmarks because Western benchmark makers follow them.

Whatever they deem as the next big thing ends up being emphasized in the public benchmarks.

Turns out that even crypto's rules only apply if you're poor.

I was a little suspicious last time, but even moreso this time. The slant their scores take, and the release timing is very suspect. On the other hand, new models drop on a near daily basis, so maybe there's no way to appear entirely fair.

An issue I think all benchmarks suffer from is regarding precision. What does a one point difference mean? How many points of difference is needed to translate into a meaningful distinction between models?

Among fellow humans, we often find ourselves picking what we consider to be most intelligent and then designing the tests around that. I've noticed a similar thing in AI, where whatever OAI/Anthropic decide to focus on improving becomes the new measure of intelligence. With their latest models, this seems to be 3d modeling/game making capabilities. Unsurprisingly, because that isn't what the Chinese companies have been focusing on, they appear behind on "frontier" reasoning tests. On top of that, older tests end up being discarded for being saturated. Makes me think that maybe unintentionally, the methodology used for these scores might be biased towards Western companies.

This would also fit with the widely discussed decline in writing quality from Opus and the worse readability of code produced by one of OAI's recent models, despite benchmarks continuing to show progress on all fronts.

Plus, if they're going to update the benchmark so frequently to discard saturated components, is there even a research-related point to keeping it private?

Out of curiosity do you happen to use a shell other than bash? I've noticed that all models, both local and cloud ones, tend to make much more mistakes if using, say, zsh. It is a frequent source of frustration for me.

Interesting report, qwen3.8-fn does seem pretty prone to hallucinating user inputs and I recall the posts about attempting to access non-existent alibaba cloud urls too. Haven't tried the other models in the gen to know if they're all like that.

To me it feels a bit extreme to jump to nefarious reasoning from this, but I think the overall conclusion is fair, nefarious or not, this would still be something certain users would need to be aware of. I wonder what the solution would be though... besides requiring all network access to be user approved?

I've been pretty satisfied with OpenCode 2

The point of having advisors was supposed to be that they would help the elected person make up for lack of knowledge, but it doesn't seem like that pans out in reality. Either the advisors are just as lost, or they are largely ignored.

System prompt changes help, but ultimately they don't do a great job of overcoming the model's learned tendencies in my experience.

My reasoning is that if something is argumentatively incorrect, it is much easier to drill down to its misunderstanding and correct it. On the other hand, if it's sycophantically incorrect, it'll just change its "mind" while potentially maintaining the misunderstanding, leading to increasingly strange behavior.

I think I'm pretty used to my tools making me want to throw them out the window sometimes :) Even before AI that was a common occurrence from either a lack of knowledge on my end, or many software people having no regard for their users' time.