Comment by amelius

Can't you take a broadly trained model and throw away useless knowledge (like foreign languages, and 99% of wikipedia) and still end up with a model that is as intelligent and useful as the original but much smaller?