Comment by machinationu

the issue is there is very little text before the internet, so not enough historical tokens to train a really big model