Comment by cpdomina

To train small gpt-like models, there's also aitextgen: https://github.com/minimaxir/aitextgen