Next token prediction is still how pre-training works, but that's a relatively small part of the whole training run nowadays. Most of the time is spent in post-training reinforcement learning scenarios, where models are scored based on an evaluation of their ability to complete a task, not whether any specific token correctly predicts a known next token.
Generally, anyone talking about next token prediction being some limiting factor of LLM capability is a good reason to ignore them, as they haven't updated their understanding of LLMs since 2023 or so.
Generally, anyone talking about next token prediction being some limiting factor of LLM capability is a good reason to ignore them, as they haven't updated their understanding of LLMs since 2023 or so.