Recent work from Meta uses AI to automatically increase test coverage with zero human checking of AI outputs. They do this with a strong oracle for AI outputs: whether the AI-generated test compiles, runs, and hits yet-unhit lines of code in the tested codebase.
We probably need a lot more work along this dimension of finding use cases where strong automatic verification of AI outputs is possible.
We probably need a lot more work along this dimension of finding use cases where strong automatic verification of AI outputs is possible.