Comment by EnPissant

I thought MTP wasn't very useful on MoE models because the expert overlap for 2 tokens was too small.