I'm not sure if this really says the truth is more complex? It is still doing next-token prediction, but it's prediction method is sufficiently complicated in terms of conditional probabilities that it recognizes that if you need to rhyme, you need to get to some future state, which then impacts the probabilities of the intermediate states.
At least in my view it's still inherently a next-token predictor, just with really good conditional probability understandings.
At least in my view it's still inherently a next-token predictor, just with really good conditional probability understandings.