Comment by anuarsh

There's one more exciting thing about Qwen3-next (except, efficient MoE architecture and fast linear attention) - MTP (Multi token prediction). It is the additional layer that allows generating more tokens without the need to go through the model again. I'm trying to make it work, but unsuccesfully yet. Maybe someone could help me with it - https://github.com/Mega4alik/ollm/blob/dev/src/ollm/qwen3_ne... (dev brunch). Take a look