Comment by ilc

To compare a 1 bit quant to the full fat model is misleading.

Honestly this model people at home can tinker with, if you have a big enough Mac. Maybe 4 Strix Halo/DGX Spark, and then at 1 bit quant? Nah.

Use the right sized model, for your hardware. You'll get better results.