To me, this is a tell of human-involvement in the model solution.
There is no reason why machines would do badly on exactly the problem which humans do badly as well - without humans prodding the machine towards a solution.
Also, there is no reason why machines could not produce a partial or wrong answer to problem 6 which seems like survivor bias to me. ie, that only correct solutions were cherrypicked.
There is no reason why machines would do badly on exactly the problem which humans do badly as well - without humans prodding the machine towards a solution.
Also, there is no reason why machines could not produce a partial or wrong answer to problem 6 which seems like survivor bias to me. ie, that only correct solutions were cherrypicked.