Comment by valine

I’ve heard the argument made that LLMs are simply models overfit on the entirety of the internet. Double decent is a good counter point to that claim, it suggests something more interesting than simple overfitting is happening with large parameter counts.