Author here. Spent the last few weeks chasing down the AI-attributed attack cases that made the rounds this year, including the Mexican government breach, the "vibe hacking" story, the Algerian amateur. Basically trying to work out whether hacking is impacted by broader AI adoption or whether the press was running ahead of the evidence.
On one side, Daniel Stenberg ran the gated Anthropic frontier model against curl on May 11. Five "confirmed" findings, one low-severity CVE after triage. His words: "the big hype around this model so far was primarily marketing." Stenberg is not a guy who hedges, and curl is not a toy codebase.
On the other side, there's SCONE — Anthropic's own December 2025 benchmark. Agents exploited 19 of 34 post-cutoff smart contracts, 55.8% success, $4.6M in simulated funds at an average API cost of $1.22 per contract. The comparable number 12 months earlier was about 2%.
Looks like agents are getting genuinely good at narrow, well-scoped vulnerability classes (Solidity, post-cutoff, bounded targets) and still bad at messy real-world codebases. But that's a guess and I'd rather hear pushback.
Happy to get into methodology, the spots where Chainalysis, Immunefi, and Web3IsGoingJustGreat don't line up, or specific cases. 28 references at the end of the piece.
On one side, Daniel Stenberg ran the gated Anthropic frontier model against curl on May 11. Five "confirmed" findings, one low-severity CVE after triage. His words: "the big hype around this model so far was primarily marketing." Stenberg is not a guy who hedges, and curl is not a toy codebase.
On the other side, there's SCONE — Anthropic's own December 2025 benchmark. Agents exploited 19 of 34 post-cutoff smart contracts, 55.8% success, $4.6M in simulated funds at an average API cost of $1.22 per contract. The comparable number 12 months earlier was about 2%.
Looks like agents are getting genuinely good at narrow, well-scoped vulnerability classes (Solidity, post-cutoff, bounded targets) and still bad at messy real-world codebases. But that's a guess and I'd rather hear pushback. Happy to get into methodology, the spots where Chainalysis, Immunefi, and Web3IsGoingJustGreat don't line up, or specific cases. 28 references at the end of the piece.