solidasparagus

Is doesn't even make sense, the limiting factor for Anthropic is GPU count, not profitability. They want more adoption at fixed GPU count which demands higher token efficiency.

AI native shouldn't imply idiotic. And designing your execution approach around a strategy based on a technology that doesn't exist is idiotic.

I don't think it's profit, I think it is genuine functionality. I think that they build features for their internal users while having no token concerns. And this mindset of using LLMs to solve product problems leads to useful features (e.g. agentic auto-approval). Then there is a bit of an issue when they try to ship a genuinely useful feature but now token cost matters.

Why is an LLVM release any different than a frontier model release? Well known, concrete issues get identified, evaluated against, RLVR'd on, and a new release comes out that doesn't have that issue. Or you can do that process yourself on an open-source model if you have the resources.

LLMs are not magic things that are out of your control, they are simply complicated. But there are concrete issues that are identified, people sit down and decide to fix it, build evals, run ablations, and they produce a model without that deficiency. It happens every day.

You look at the inputs to the model at the trace level to root cause context construction vs model deficiency. You collect examples of the failure cases, build eval sets, and run ablations. That's black box debugging, but systematic debugging nonetheless.

Not 1 in 1000 software engineers has the ability to debug a compiler. And you can absolutely debug agent failures, but similarly almost no does. In both cases for the mast majority of engineers, they are black boxes.

I'm not sure. That is also my experience, but I'm not completely sure that isn't my fault and the compiler comparison would be totally valid if I just used the tool better. When I actually write fully fleshed out specifications and have the LLM handle the implementation, I rarely encounter bugs.

I don't think I've seen a Sol 5.6 or later model fail at a task with a verifiable outcome.

It's when I try to do too little of the defining that I get LLM bugs. In which case maybe programming hasn't changed that much, LLMs really can be reliable compilers, and we're all just being lazy.

Neither are compilers? But you still trust your JIT compiler. Determinism is unimportant, likelihood of correctness is what matters. Plus agents are deterministic, no?

How does it differ other than that people aren't currently choosing to write clear, well-defined high-level specs and feeding that into their agent?

Complete comprehension has never been a thing. Unless you were working solo with no dependencies, you never completely understood what was being shipped. I don't think an npm import vs an LLM implementation of a well-scoped, well-abstracted library is any different.

At agentic coding velocity, tech debt becomes obvious much more quickly. Empirical evidence is a bit of a ridiculous bar to hold for software development - we are as quantitatively blind now as we have ever been - but at least for me the anecdotal evidence is very clear. Today's agents hill-climb and their software architecture is pretty reliably bad without a human in the loop.

And it's debatable how much better they will get quickly. Verifiable rewards for maintainability of software is, I think, possible but very hard.

0.5 tok/s I think is their claim. Basically skynet

You should read the report, it is interesting technically if you ignore the fear-mongering. It distributes the model's layers over multiple compromised devices.

This technology has surpassed the the world's best mathematicians at the (explicitly and openly stated) goal by which they measure and reward mathematical progress. Such an advancement that it seems to have thrown the entire field into existential self-doubt. And then someone sits at their keyboard and says it is just a fuzzer.

I think the answer is literally no, they don't have the ability.