Comment Comment by DarthNebo JM2Clood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM. parent DarthNebo 3 years ago
lood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM.