Comment Comment by reidrac > on a decent speedBut you said 7-9 tokens/second, that's not a decent speed. I'm not an expert by all means but in my local experiments, less than 12 to 16 tps is too slow. parent reidrac 3 months ago
But you said 7-9 tokens/second, that's not a decent speed. I'm not an expert by all means but in my local experiments, less than 12 to 16 tps is too slow.