Comment by sheo

As said in the article, all of the providers run different runtimes with proprietary, sometimes untested and weird optimisations

There are multiple types of quantisation: weights and KV Cache. Quantizing kv cache can drastically hurt performance