Comment by ahmadyan

even in pre-training, usually 30%-50% is code these days.