Systemerror7A69

Irregardless of wether or not Mistral Large 4 is competitive or not, I do not trust the AA numbers anymore, for various reasons. Them changing their numbers around so much recently brings the entire validity of this in question, not to mention if such a combined number is even important at all. The combined "intelligence" score of AA includes areas I do not care about or need at all.

Yeah, absolutely true. And I am guilty of simply using Mistral as a GLM subscription myself. I just really like the GLM 5.3 model.

I will see where Chonk is better though, since you have both options in one subscription.

It's gonna be interesting to see the final version, since this is a preview model still. I'm not completely sure as to how this works, but from what I gathered it will be further trained during this period, so it might improve a bit.

Since cost is so much higher than deepseek though it really seems like the biggest reason to use it would be "in the mistral subscription" for personal use or "because you need to abide by european data privacy laws" for companies.

Mostly I'm happy they're still in competition, as I mentioned already elsewhere.

Well, Models are hard to just distill down to a single benchmark number, or even a few across multiple different benchmark - they all behave and work slightly differently and you can find some who work or feel better for you and your use-case specifically.

But you're right, Mistral Large 4 seems a much more expensive but overall weaker model across the board.

The reason to use them would be, well, because they're...Mistral. They're in EU, france is large in nuclear energy so the energy cost of training has much less of an environmental impact, the data protection laws are very good in EU. This can really be a deciding factor for many EU companies, and I think it's good that they now have something which is at least semi-usable. (apparently)

It's also good that other options exist, even if they are worse. Deepseek is kind of like a large specter looming over every model release, because it's just SO cheap and good. That doesn't mean we should abandon every other model.

Still, the model is not really competitive on price and abilities alone. I'm mostly happy to see Mistral still in the fight.

This was also my question, and I'm surprised this hasn't come up more prominently in the discussion here at HN.

I don't use Anthropic products at all, so I was not really sure myself but apart from the models they pretty much only have Claude Code afaik.

And now they're publishing a blog post about writing their own vcs because they have 30000 commits a day. For what? Isn't Claude still based on react?

Do they need this many commits just for claude? Or is that about the model, do they develop that much extra internal tooling?

I admit it is very much possible I am wrong, but without further information this seems a bit like a red flag to me.

I swapped to hax after the atuhor talked about it here on Hackernews, it really is very much a huge recommendation. Most importantly for me, it has been the only harness who respects the XDG specification and does not ask me to install by curling into sh. From my previous to harnesses, pi and maki, neither did both.

On top of that, it also feels very much like a normal CLI tool, which is a big plus for me.

It does not support plugins or MCP, which could both be a big negative for many people, but for now I was able to find a good terminal alternative for everything I needed.

Just a huge recommendation from my side as well, not affiliated in any way myself.

I feel like the "good enough" argument isn't about how big the gap between models is but about how good they are at solving the tasks at hand.

The capabilities of all models increasing so much all the time means there are simply less and less tasks you need a frontier model for.

Even if Opus 5.5 is 500x better than Deepseek, if deepseek can solve all my problems, why do I need to pay for more?

Not the person you asked but I can answer that from my perspective:

I am always on the lookout to automatate as much boring, everyday tasks. A Smart Fridge like that would allow you to generate shopping lists and reduce food waste by alerting you to possible soon-to-be-spoiled food.

A bit like what roombas are doing for cleaning. You still need to do it yourself a bit, but they help.

Not necessary, not even a gamechanger but another neat home automation. Would be nice, sad to hear they don't work well.

I'm not sure I fully agree with the article either, but I think that comparison is kinda nonsensical, over generalized and just handwaves a bunch of points and arguments about the article.

I don't think this brings your point across at all. If you disagree with the article, just saying "no, it's a tool, like coffee" does not meaningfully engage at all with any of the points made.

This is just speculation on my part, but LLMs work best when they get immediate, verifiable feedback on their task, and the kind of physical optimizations they mean might not give that to LLMs.

I'd really love to see more studies about effectiveness of AI in general. As in, what works best and how to use it and such.

Because I feel that the technology and space is - so - hyped and fast moving that a lot of cultish feeling rituals seem to pop up, none of which are backed by evidence. Anthropic openly recommend giving the agents.md file an architectural overview of the code, and the one time this was studied they found the opposite - that the agents.md file is best for concrete commands about how to build stuff and such, and - not - huge overviews. This was, and still is, the official recommendation from Anthropic as far as I can tell.

And then there are the benchmarks, how feel vague and not concrete, and everyone kind of knows they're not the best cuz you can't just assign these tools one fixed number ( for multiple reasons ), but everyone still looks at them and compares them.

People share skills and superpowers and plugins and mcps and very, very few of them have and kind of proof they do much at all.

It all feels a bit weird to me, and I've been on the lookout for exactly these kinds of studies more lately, because I think having this research, even if not done on the exact newest models or not the exact, newest thing, are still - vastly - superior to the alternative.

I have to say, I am starting to hate this line of reasoning. Yes, LLMs move extremely fast and a lot of improvements are done in a short amount of time.

And there might be a point to these arguments, vaguely. However:

There never seems to be - any - kind of counter example or reasoning behind the rationale. You have an in depth and empirical study, done by researchers who, frankly, now their shit (most of the time)

And on the other hand a random internet comment saying "nope" because...the models aren't the latest.

If the latest models really would make a difference, you should at least provide some kind of evidence towards that. As it stands though, every time these comments come up this is missing.

There seems to just be a vaguely defined understanding that "everything changes all the time, and nothing you ever research is transferable to state-of-the-art models"

Which brings me to my second point about these kinds of arguments:

LLM models often - aren't - fundamentally different. Yes, they are vastly more capable. And yes, there are emergent properties. But at their core, they function very much similarly. And for quite a while now, there have not been any of these drastic changes we saw when LLMs first become "good enough" for agentic coding.

I am tired of dismissing empirical evidence and studies every. single. time for reasons without evidence and seemingly a vague sense of "no, but my model is different"

AMD 7900 XTX with Vulkan here as well, wasn't faster on my test either. Might be much different on Nvidia though.

I assume the limit for me is memory bandwith, as the 7900 XTX has the same bandwith as the 3090 from what I can gather and I already reached ~60 t/s with Unsloth. Those would fit with the numbers Byteshape has for their cards.

4090 and 5090 have much higher bandwith apparently, so on those cards you can probably get much more out of the kinds of performance improvements they are doing.

My approach to this problem is to just...not try them all.

As long as the model you're using solves the problems you have to your satisfaction, there is no need to try any other models, except for financial reasons maybe.

So I start with a relatively cheap model (GLM 5.3 flash for me) and as long as it accomplishes the task (it did so far) I don't have to change. And even if it can't do something, the first thing I change is see if I can give it more tools or better context (useful even if I switch models later) or trying a different approach to the problem.

If google/gemma-4-31b works, you don't need to overthink it.

So, alignment does need to be taking seriously, you're right.

But keep in mind this is a report from OpenAI about OpenAI, who have a financial incentive to present this in a certain light. Take these things with a grain of salt.

This does not mean that models are now self-aware.