Comment by wat10000

Right. But that's never what the discussion is about. This is the first time I've seen someone say, "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." I have seen many, many people say "it's not just a next-token predictor because there's all this smart stuff going on inside the model." Even here, in the comments for an article that's all about the former, most of the objections to "next-token predictor" are of the latter form.