jfaganel99
Joined 6 years ago
Karma
76
Submissions
33
Member since
Jun 1
Always building something
View on Hacker NewsJoined 6 years ago
Created an open-source benchmark for code security scanners and ran a bunch of them along with LLMs on real vulnerable code. Fable 5 is on there also as of yesterday, but that's the gated public model. The one we all wants to see is Mythos 5, and it's locked to a handful of vetted orgs.
So does anyone here have access to Mythos 5? And can run it against the benchmark.
Would genuinely like to see what it scores and at what cost.
Updated benchmark results published here also. BTW, with v002 we are consistently hitting 75+