nilkn

Autonomous software and hardware verification. They're just getting started. It's not just Axiom: Logical Intelligence, Harmonic, and a new one that just launched (&Truth). I suspect the business case is addressing the global cybersecurity crisis that AI is creating.

Making it cheap to do is probably exactly what they're trying to do. Formal proofs for verification take 10-20x as many lines as the core functional code being verified. Nobody's going to want to pay for 10-20x as many tokens at frontier pricing for verification unless they have to or it yields tremendous benefits. So getting the token price down as low as possible for a prover is probably a core part of the value proposition.

I understood them to be arguing that AI models are taking away the joy of certain things like mathematics but not producing commensurate value for humanity. We have a lower bound on twin primes, but that doesn't improve life for anybody, and the cost of getting the bound was removing human involvement from a lot of mathematical research, which was a source of joy for some portion of the human race. If all of this leads to curing major diseases without an AI takeover, it will have been worth it. But if it doesn't, then the ends don't necessarily justify the means.

You're trying to get me to make a claim stronger than what I'm making. The claim I'm making is that capabilities have moved since the paper's data (countless benchmarks support this), and it's not possible to measure the end-to-end effect of that inside industrial applications with complete access, adoption, and internal connectivity at the same pace. Therefore, the study is fundamentally out of date and conclusions about the present or future cannot be drawn from it unless you assume that technological progress cannot have affected the relevant outcomes, which is a strong assumption you'd have to back up separately. Note that the paper's model ties end-to-end outcomes to generated code quality and correctness which is precisely one of the variables that continues to move.

What the study actually shows is that organizations that had very little time to absorb older generations of a disruptive emerging technology experienced an indeterminate change in output (at the 12 month mark, anywhere from down 13% to up 25% -- the paper can't say where reality falls). It certainly doesn't establish any ceiling.

> I mention this because I've been hearing "Yeah, well, that opinion/study is completely invalid because it used a six month old tool. Today's release of SOMNOS 6.9 (Pinnacle Snoozytime) has totally changed the game!" every six months for what feels like the past five years now.

Given my own conclusion that it's not possible for conventional academic research to accurately snapshot industrial adoption of this technology, I would expect this pattern to continue for a while until either academia becomes much faster or technology progress slows down (or both). This just isn't yet a situation where you can rely on published papers telling you what to think -- you still need to mix in a bit of your own instinct and experience and extrapolation to get an accurate up-to-date read, even if there's some prediction error as a result.

I don't, actually. I can simply point out that its data is out of date, and then the numbers themselves are called into question without other numbers to compare them to. And that's what I did: I shared two arguments in two comments that converge on the shared conclusion that the paper's data is not up to date (or, more precisely, cannot be up to date).

I have my own experience to go off of, but this can also be inferred easily. There are many large corporations still working on "digital transformation" from 10+ years ago. There's no way such companies have fully onboarded and integrated frontier AI agents throughout their workforces with high adoption. The paper itself reports low adoption even after conditioning on access.

Everything did change after late 2025, but there are tiers of usefulness. Late 2025/early 2026 was the first time agents became clearly useful for personal projects. That's distinct from immediately having economically valuable end-to-end autonomous workflows in teams of median talent, which many would suggest is the threshold we've just recently crossed.

The data is still mostly 2025 and the 2026 data is mostly from the first few months, and most production deployments (at least in, say, the Fortune 500) lag the frontier by at least several more months (or much more). I suspect it's just not really possible for academic research at its conventional pace to offer an up-to-date perspective on this topic right now.

It's still the world's largest prison: most North Koreans would be killed on sight or imprisoned for even attempting to leave (i.e., escape). It also has millions of slaves, in fact the highest rate of modern slavery in the world (approximately the same as the US in 1860 before the Civil War). Kim Jong Un executed his uncle in 2013. Privilege or comfort, where it exists, can be stripped by force from an entire family at once for any reason. So, yes, Kim Jong Un is not a literal god on Earth -- there are balances he must maintain within the country to keep his power. Not every North Korean is a slave. You're not wrong about those things. But I feel your description is coming dangerously close to portraying it as a relatively normal country, which it most certainly is not. It is a prison and slave state, one of the most severe in history, enforced via extreme brutality and shielded by nukes against those who would save its people.

Like when it happened in 1637?