Comment by midland_trucker

> Dataset size is not relevant to predicting the loss threshold of LLMs. You can keep pushing loss down by using the same sized dataset, but increasingly larger models.

Deepmind and others would disagree with you! No-one really knows in actual fact.

[1] https://www.deepmind.com/publications/an-empirical-analysis-...