Comment Comment by frozenport M2 macs can do it: https://twitter.com/junrushao/status/1681828325923389440in practice 10 tokens per second is kinda annoyingly slowmost local people would opt for a smaller 7b model parent frozenport 2 years ago
in practice 10 tokens per second is kinda annoyingly slow
most local people would opt for a smaller 7b model