Comment Comment by itsTyrion > "wanted to run glm-4.7-flash:q8_0" > q8_0a well made (as in, unsloth) smaller quant will help a good amount here, without a notable reduction in performance or increase in perplexity parent itsTyrion 7 months ago
a well made (as in, unsloth) smaller quant will help a good amount here, without a notable reduction in performance or increase in perplexity