Hey there! Definitely good points here. I think some of these are covered in the benchmark, for example, false positives requiring investigation is equivalent to (1-precision), bugs that still escape into production is (1-recall), and real defects is recall.
Our goal with Mo is to make maintenance effort as low as possible while still maintaining quality. One thing we've been talking about separately is running a benchmark where all of the agents have virtually unlimited time (or something really high like 24h) and they need to find as many bugs as possible. Is that more similar to what you're looking for?
Our goal with Mo is to make maintenance effort as low as possible while still maintaining quality. One thing we've been talking about separately is running a benchmark where all of the agents have virtually unlimited time (or something really high like 24h) and they need to find as many bugs as possible. Is that more similar to what you're looking for?