Comment by Gerard22Aug

Yeah, there is latest and greatest trend: 0-bit quant. Absolutely outstanding performance, can produce as many tokens per second as you terminal may sustain with zero load on unified memory / GPU. Real winner!