Comment by didroe

I think you're conflating prediction and training. The models are auto-regressive so they always predict the next token.