aramend

LLMs being described as system 2 thinking here is a semantic shift I have not encountered before.

LLMs are also a deep learning approach. Output, as slow as it is, still comes from weird latent spaces. In AI I always took System 2 to map more to symbolic approaches, or at least when explaining symbolic AI to someone who has heard of deep learning thinking fast and slow was a good comparison to draw on.

> For ARC-AGI challenge, we start with all input-output example pairs in the training and the evalua- tion sets ... At test time, we proceed as follows for each test input in the evaluation set: ...

Very often I see people misuse the ARC-AGI data when training. The input examples in the evaluation set are not intended for training your AI system. It is a downside of ARC that its data is (somehow?) complicated enough for the clever people building AI systems to miss the point, and people report and compare results as a single percentage where the data mix used for training may not make the comparison applicable.

From the linked page

> Why say crypto if it's not analyzed?

> It's intention is to be a cryptohash so it should be analyzed to verify and understand if it's secure and how secure. The inclusion of these words is to indicate the intention and invite people to analyze, break, or improve it.