Comment by jephs

Every frontier model is MoE these days! Flash or otherwise. Sparsity just gets you more bang for your buck.