Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong:
1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector.
2. What you see when you use an LLM is not next-token prediction directly next to the prompt, but instead following a block of varying length of next-token prediction that happened to make progress on the problem in your prompt, and which just summarizes the results.
1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector.
2. What you see when you use an LLM is not next-token prediction directly next to the prompt, but instead following a block of varying length of next-token prediction that happened to make progress on the problem in your prompt, and which just summarizes the results.