Comment by scrlk

If you view LLMs as lossy data compressors, this exactly what you'd expect.