Comment by dgant

Top bots are in the ballpark of 50kloc. Even large LLMs are not so large that they can memorize all the world's data like that. And all the fine-tuning that compression loses comprises most of the work in the first place.