Comment by brucethemoose2

Reading between the lines, it sounds like some of the speedup comes from VRAM savings on an otherwise close to full GPU?

This is definitely cool and needed, but it might not be so dramatic running 3-5 but quant on a less full GPU.