tob_scott_a

When people question the efficacy of using AI for security work, they're often asking the wrong questions. (I blame the initial marketing hype around Claude Mythos for much of the unhelpful discourse.) And I don't just mean they're asking their AI the wrong questions, either.

Two problems that often arise when using LLM-based vulnerability assessment methodologies for a codebase are false positives and severity inflation.

As far as I can tell, false positives occur when the underlying language model pattern-matches a snippet of code with vulnerable, risky, or error-prone samples in its training data and/or context window. (Many SKILL.md files you find on GitHub are piles of Markdown describing anti-patterns, and occasionally TOML files describing roles the agent can assume when interpreting that Markdown.) It might be gesturing towards a real problem, but in practice, the signal-to-noise ratio can be extremely low without validation steps or expert feedback.

I don't quite understand the source of the usual severity inflation observed with LLM output, but I also haven't seen the training data, so it could just be a maladaptive behavior from the source material.

Regardless, the two problems are a symptom of the same systemic flaw: Insufficient rigor.

If you decide the work is done once you have "a result", you're doing yourself and your audience a disservice. It's imperative to have explicit mechanisms to ensure that the result is: a) real, b) relevant to the threat model of the application in scope, and c) measured appropriately in terms of both severity and difficulty. If you build the right mechanisms, you solve both symptoms in one fell swoop.

The post-patch-validation skill mentioned in the blog post was created as one of several mechanisms to ensure that the final output (i.e., what humans review) is actually worth anyone's time. We have other interesting AI experiments that I plan to open-source in the coming months (or in early 2027 at the latest).

Happy to answer any questions and hear any feedback anyone has on this skill.

If you can't write it down, why would you expect it to be universal and enforceable? Different cultures exist and have different opinions on what "decency' means, after all.

A security researcher's ethical obligations are to protect users over vendors (barring any contractual agreement in place). From what has been discussed in this thread, they meet that bar.

Sure, they could have gone the extra mile to ensure the distros were in a good place to patch before they published the exploit. That's a kindness you can wish for, but don't disparage them for not going that extra mile. It's a bonus.

It's also possible that it simply didn't occur to them to do so this time. There's certainly lessons to be learned either way. I don't know that the right lessons will emerge from hostility.

> I think that attitude vastly underestimates the complexity of a typical TLS implementation

If you ever get the impression that I'm underestimating the complexity of a typical TLS implementation, I promise you that I'm not. I speak to improvements, not panaceas.

Until the end of last year, I was one of the security engineers that the s2n team at AWS consulted on potential security issues. You will never hear me say anything will magically fix all our problems. Especially with TLS.

However, Rust does bring a lot to the table, so I feel I'm allowed to be excited about not reviewing another X.509 library written in C.