Because it's a set of puzzles on a 2D grid. We don't live on a 2D grid so it's already on the wrong track. A set of puzzles for a 3D sphere wouldn't get us any closer to AGI either but at least it would be a more realistic representation of the world and how a general purpose problem solver should approach reality. Even Minecraft would be a better test and lately people have started testing LLMs in virtual worlds which is a much better test case than ARC.
Insofar as ARC is being used as a benchmark for code synthesis it might be somewhat successful but it doesn't seem like people are using code synthesis to solve the puzzles so it's not really clear how much success on ARC is going to advance the state of the art in AI and code synthesis according to a logical specification.
Insofar as ARC is being used as a benchmark for code synthesis it might be somewhat successful but it doesn't seem like people are using code synthesis to solve the puzzles so it's not really clear how much success on ARC is going to advance the state of the art in AI and code synthesis according to a logical specification.