Comment by nee1r

Definitely much less redundancy, this was definitely a tradeoff we made for pretraining data and cost.