Comment by MacsHeadroom

RAM, not VRAM. The model is fully loaded into (regular, system) RAM then quantized while being loaded into VRAM.

So you still need enough system RAM (or RAM+Swap) to load the unquantized model.