Comment by numba888

github has a bunch of them for years, the most known from Andrej Karpathy:

https://github.com/karpathy/nanoGPT

some other have MoE implemented.