Comment by slowin

That's a proof of concept created by a small startup. Obviously Intel or NVIDIA could do better. Given that it's ~50k tokens per second, even a lesser model opens up an entire universe of use-cases that we currently don't have access to, and that's locally.