p0u4a

> if software knows in advance which data chunk (expert) it'll need for the next token, it can load that in parallel with computing current token

You could actually use the model's MTP head to make a ~decent prediction on what experts would be activate in future tokens and preload them

more of stuff like this on the web please

I have no idea what's going on but it looks really cool

How much does training such a model cost?

Sadly they don't teach Show Don't Tell in the lectures

Aaaand we’re back

Edit: I was hosting this on Vercel free tier, and since this post made it to the front page, I've hit Vercel's quota limit. I will work on getting around this.