Comment by abeppu

I think you're basically saying that ARC-AGI doesn't achieve a goal that _it didn't set_. The point of ARC-AGI is not to benchmark LLMs specifically. The point is to measure fluid intelligence in a way which supports comparisons between models and between models and humans. It's not the obligation of the test to be tailored to the form of model that's most popular now.