Comment by mrtsepelev

Congrats on launch! Tried it on the gemma-4-26b-qat-4bit model. Was indeed faster on token generation then on oMLX (82.8 tok/s vs 76.5 tok/s), but the prefill time was ~2.6x slower (709 tok/s vs 1843 tok/s). Don’t use any acceleration on the oMLX. Macbook M5 Pro, 48 gb