Comment by chrisjj

> Next token prediction is still how pre-training works, but that's a relatively small part of the whole training run nowadays

Only if you call 90% small.

No-one is claiming <50%.