Comment by demibabs

How is solving unsolved problems a good eval? Once a problem is solved and released, you can’t evaluate a future model’s ability to solve it.