I'll be honest that I've never actually considered tokens per second as something to focus on for my projects, I'm much more concerned with quality of the output then quantity.
Is 500 tok/s on Gemma 7B a gamechanger? or is this more just an advertisement for mystic.ai?
Is 500 tok/s on Gemma 7B a gamechanger? or is this more just an advertisement for mystic.ai?