It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.
I've taken a vibe-coded project pretty far, with even better testing than most other things: the main factor is that it's big on falsifiability, i.e. it has to see a test for a new feature actually fail before it considers the test any good. Not just the red/green from being originally unimplemented, it does a mutation test to ensure it flips back to red. But wow did it ever make that test framework complicated, byzantine, and ultimately brittle until I stepped in and forced a consolidation and simplification pass (twice now). Part of the complexity came from it being a polyglot monorepo, but I've found even in other places, you have to follow an expand-contract cycle of development when vibe-coding. Not all that different from any other system you've built up organically, just on a much faster cycle.
But back to my original point, none of this really applies to the pgrust benchmark, which was obvious and blatant cheating on the human's part.
I've taken a vibe-coded project pretty far, with even better testing than most other things: the main factor is that it's big on falsifiability, i.e. it has to see a test for a new feature actually fail before it considers the test any good. Not just the red/green from being originally unimplemented, it does a mutation test to ensure it flips back to red. But wow did it ever make that test framework complicated, byzantine, and ultimately brittle until I stepped in and forced a consolidation and simplification pass (twice now). Part of the complexity came from it being a polyglot monorepo, but I've found even in other places, you have to follow an expand-contract cycle of development when vibe-coding. Not all that different from any other system you've built up organically, just on a much faster cycle.
But back to my original point, none of this really applies to the pgrust benchmark, which was obvious and blatant cheating on the human's part.