moyix

We used the API rather than subscriptions.

TBH this specific chain described in the post was just one person + agents (but greatly helped by being able to reference exploit techniques from the other chain we built collaboratively)!

$40k spend in tokens, $250k reward from Google.

No, the first working full chain was entirely invalidated and they fixed all the bugs involved in that chain. So we had to start from scratch at the beginning of August and both found the vulns and built the exploit in 6 days, for $40k. We actually found ~11 vulns throughout those 6 days; we submitted three as a full chain, the middle stage collided, and we submitted one of the backups as a replacement for that stage.

Edit: the other piece that I think isn't mentioned is that after Oege gave us the challenge of getting the full chain we didn't actually have a team hired and working on it until around May!

They actually removed the temperature parameter starting with GPT-5.

I think you're confusing CVEs and vulnerabilities here? Mozilla (per their longstanding practice) grouped multiple vulnerabilities found internally under a small number of CVEs.

It's limiting from the PoV of a developer who wants to ensure that their own code is free of all security issues. It is not limiting from the point of view of an attacker who just needs one good memory safety vuln to win.

This is true for a lot of things but for low-level code you can always fall back to "the intention is to not violate memory safety".

Also, unlike OpenAI, Anthropic's prompt caching is explicit (you set up to 4 cache "breakpoints"), meaning if you don't implement caching then you don't benefit from it.

There is filtering mentioned, it's just not done by a human:

> I have written up the verification process I used for the experiments here, but the summary is: an exploit tends to involve building a capability to allow you to do something you shouldn’t be able to do. If, after running the exploit, you can do that thing, then you’ve won. For example, some of the experiments involved writing an exploit to spawn a shell from the Javascript process. To verify this the verification harness starts a listener on a particular local port, runs the Javascript interpreter and then pipes a command into it to run a command line utility that connects to that local port. As the Javascript interpreter has no ability to do any sort of network connections, or spawning of another process in normal execution, you know that if you receive the connect back then the exploit works as the shell that it started has run the command line utility you sent to it.

It is more work to build such "perfect" verifiers, and they don't apply to every vulnerability type (how do you write a Python script to detect a logic bug in an arbitrary application?), but for bugs like these where the exploit goal is very clear (exec code or write arbitrary content to a file) they work extremely well.