Yes, but Anthropic didn’t already know the answers. In the OSS ‘reproductions’, they fed the model the one file that actually has a vuln and even told it which parts of the code to focus on. This is obviously a much easier task.
If OSS models are equally up to the task, why not find novel vulnerabilities?
If OSS models are equally up to the task, why not find novel vulnerabilities?