- Aug 6: Off By 1, the security lab of 1Password publishes their blog post (current title: Why AI-generated vulnerability patches still require expert human review)
This article gets some specialized and mainstream press coverage
- September 15 : Trail of Bits publishes a rebuttal on their blog 1Password's AI patching benchmark is misleading
- September 22: Suha publishes the current blog post
The main idea in the rebuttal is that the headline number (26 percent only of the AI generated patches are flawless) is misleading because the experiments were not well designed.
- Aug 6: Off By 1, the security lab of 1Password publishes their blog post (current title: Why AI-generated vulnerability patches still require expert human review)
This article gets some specialized and mainstream press coverage
- September 15 : Trail of Bits publishes a rebuttal on their blog 1Password's AI patching benchmark is misleading
- September 22: Suha publishes the current blog post
The main idea in the rebuttal is that the headline number (26 percent only of the AI generated patches are flawless) is misleading because the experiments were not well designed.