Comment Comment by elephantum I see, in order to benefit, model has to be quantized. It is not super clear which kinds of quantization are supported. Both Fp16 and Int8? parent elephantum 5 years ago
