The first question I still have is what happened to core knowledge priors. The white paper that introduced ARC made a big todo about how core knowledge priors are necessary to solve ARC tasks but from what I can tell none of the best-performing (or at-all performing) systems have anything to do with core knowlege priors.
So what happened to that assumption? Is it dead?
The second question I still have is about the defenses of ARC against memorisation-based, big-data approaches. I note that the second best system is based on an LLM with "test time training" where the first two steps are:
initial finetuning on similar tasks
auxiliary task format and augmentations
Which is to say, a data augmentation approach. With big data comes great responsibility and the authors of the second-best system don't disappoint: they claim that by training on more examples they achieve reasoning.
So what happened to the claim that ARC is secure against big-data approaches? Is it dead?
So what happened to that assumption? Is it dead?
The second question I still have is about the defenses of ARC against memorisation-based, big-data approaches. I note that the second best system is based on an LLM with "test time training" where the first two steps are:
Which is to say, a data augmentation approach. With big data comes great responsibility and the authors of the second-best system don't disappoint: they claim that by training on more examples they achieve reasoning.So what happened to the claim that ARC is secure against big-data approaches? Is it dead?