Comment by rose_ann_

MoE model with 52 billion activated parameters means its more comparable to a (dense) 70b model and not a dense 405b model