I'm wondering how long till we see the first exploit that goes like this:
* A bad actor finds a real, theoretically severe but not easy-to-actually-exploit vulnerability in some popular software. E.g. it's an exploit that can get you pwned 100% with RCE, but only if you are using it in configuration X on platform Y with kernel version Z on a Tuesday.
* They design a patch that closes that vulnerability but subtly opens a more dangerous one known only to them. They privately test their patch against LLM reviewers, tuning the approach until the introduced vulnerability typically escapes detection by automated review.
* They "responsibly disclose" the original exploit and helpfully share the patch. In the hurry to fix the original exploit, the patch itself gets only cursory (claudy, codexy) review. By now this has happened so many times that just applying whatever code changes ClodSed were GTG is the status quo.
* When the new vulnerability is eventually discovered, they can plausibly say "oh my agent didn't catch that", which is true. In the meantime, they use the new exploit that only they know about.
* A bad actor finds a real, theoretically severe but not easy-to-actually-exploit vulnerability in some popular software. E.g. it's an exploit that can get you pwned 100% with RCE, but only if you are using it in configuration X on platform Y with kernel version Z on a Tuesday.
* They design a patch that closes that vulnerability but subtly opens a more dangerous one known only to them. They privately test their patch against LLM reviewers, tuning the approach until the introduced vulnerability typically escapes detection by automated review.
* They "responsibly disclose" the original exploit and helpfully share the patch. In the hurry to fix the original exploit, the patch itself gets only cursory (claudy, codexy) review. By now this has happened so many times that just applying whatever code changes ClodSed were GTG is the status quo.
* When the new vulnerability is eventually discovered, they can plausibly say "oh my agent didn't catch that", which is true. In the meantime, they use the new exploit that only they know about.