Comment Comment by cgdl Very cool. For the INT4 QAT model, what is the recommended precision for the activations and for the key and values stored in KV cache? parent cgdl 1 year ago
