Comment by truncate

Or, they can figure out something else out? I recall couple years ago when China didn't have enough GPUs (still don't?), DeepSeek team figured out how to train with less computing. IIRC they made Mixture of Experts mainstream and made really optimized kernels and clever use of PTX instruction set.