Comment Comment by ogrisel It appears that they reused a lot of the data preparation provided by the AllenAI team:https://github.com/allenai/OLMoEhttps://github.com/allenai/dolmahttps://github.com/AMD-AIG-AIMA/Instella parent ogrisel 1 year ago
https://github.com/allenai/OLMoE
https://github.com/allenai/dolma
https://github.com/AMD-AIG-AIMA/Instella