Comment by brandonasuncion

Btw, the quantized models are on Huggingface, so step 1 and steps 3-6 can be avoided. (The link is 3/4ths into the llama.cpp README)