Comment by mxwsn

No, there are more training tokens than parameters in LLMs. They are in the classical first descent setting.