Comment by philipkglass

What was the "vanilla post-training quantization" used for comparison? There are 22 GGUF quantization variants smaller than 16 bits per weight and I can't tell which one is being compared with:

https://huggingface.co/docs/hub/en/gguf#quantization-types

It might even mean a non-GGUF quantization scheme; I'm just an intermediate user of local models, not an expert user or developer.