I don't think the distinction between statistical correlation and understanding is as clear-cut as you're suggesting.
For example, Li et al. (2023), Emergent World Representations, showed that a GPT model trained only to predict Othello moves developed an internal representation of the board state, without being explicitly taught the rules or board structure.
For example, Li et al. (2023), Emergent World Representations, showed that a GPT model trained only to predict Othello moves developed an internal representation of the board state, without being explicitly taught the rules or board structure.