Comment by superjan

I have only passing familiarity with how LLMs work and I found it enlighting. It makes me curious how recall quality degrades when quantization is applied. It looks like it’d gets worse quickly.