Comment Comment by bloaf This bloke on huggingface documents the memory requirements for his quantized versions of popular models: https://huggingface.co/TheBlokeTl;Dr, Max ram needed depends on quant method, rough ranges are:7B models are in the 4-8GB range13B models 8-15GB30B models 13-33GB70B models 31-75GB parent bloaf 3 years ago
Tl;Dr, Max ram needed depends on quant method, rough ranges are:
7B models are in the 4-8GB range
13B models 8-15GB
30B models 13-33GB
70B models 31-75GB