Comment by jsheard

It may be obvious, but it does resurface the question of how these models are supposed to absorb post-2021 information effectively when the open internet from that point forwards is increasingly being filled with low-quality AI slurry. Internet scrapes being a representative sample of human-created media was a one-time-only deal.