Comment by username90

I think the "turing test" for code generation would be to be able to do most leetcode problems and other competitive programming problems. If you can do that you have done something amazing, and the dataset and testing for it already exists.