Comment

Comment by anushka_gandhi

No, you require BF16 model weights for this quantization to work. If you do run it on 4bit or 1.5bit, it will degrade the model and it will hallucinate a lot.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton