Comment by penciltwirler

Hugging Face's internal storage system is more sophisticated now after they acquired XetHub. They have a clever way to chunk and content dedupe the data that makes it a lot more efficient to version control large files.