It will never happen with LLMs, but we can hold out hope that it will happen someday. Humans do not become what we are by having the whole of the internet poured through our heads multiple times over. It seems almost necessary that there is some next-generation architecture that won't require as much training data.
Probably still enough that it'll stagger a home connection trying to download a torrent or something, but perhaps not so much that it is insane to also release the core data set one way or another.
It will never happen with LLMs, but we can hold out hope that it will happen someday. Humans do not become what we are by having the whole of the internet poured through our heads multiple times over. It seems almost necessary that there is some next-generation architecture that won't require as much training data.
Probably still enough that it'll stagger a home connection trying to download a torrent or something, but perhaps not so much that it is insane to also release the core data set one way or another.