Comment by lucrbvi

There are a lot of open-research on pre-training, post-training and RL data mixtures and sourcing.

I recommend checking papers from Datalogy, Nvidia Nemotron, Ai2 (Ollmo, Tulu, ...) and the recent model from Aleph Alpha if you want to learn more.