Comment Comment by elorant Is this at 4-bit quantization? And how many tokens per second is the output? parent elorant 2 years ago
