Comment Comment by sigbottle It's interesting though that Q4 seems to be enough, is there a reason that 4 bit floats are good enough for inference? parent sigbottle 27 days ago
