Comment by liuliu

MoE, especially these Chinese ones trained on low-bits already is actually quite quantizable to lower bits, unlike smaller / dense models.