danieldoesbio

What is the failure state for a pass that doesn't find a real vulnerability? Do the models report no issues or hallucinate issues that aren't real? I'm trying to run open weight local models and finding them really impressive... Just also trying to understand the cybersecurity side of all this.

Genuine question here about the open-weight models finding the same vulnerabilities as mythos thing: is it just a matter of false negatives/positives? I’ve seen a few cases where people show other models (even opus) can find the same vulnerabilities given many passes. Is there some disadvantage to the extra passes that give the claimed Mythos performance extra value (assuming it finds them in less)?