Comment by seanmcdirmid

That isn’t right. Use oMLX or something similar to serve the model with internal MTP enabled. You should get at least 40 tok/sec, but I’m not sure what hardware you are using. Dflash will mess up batching, not really worth it. You need to sift through hugging face for the right model though, and some of the MTP models are meant for rapid MLX and not oMLX.