Comment by michalpleban

Like I said above, you should benchmark quantized versions. With quantization, the same models can be ran at much cheaper hardware, but quality loss is real and this kind of benchmark is an ideal place to put a finger on it.