Comment by ALTRC0320

Hey HN, I made this simple GitHub tool to check how much vRAM you need to train or inference any LLM. It supports inference frameworks (huggingface/vLLM/llama.cpp) & quantization (GGML/bnb) I Made this after getting frustrated when I couldn't get 4bit 7b llama to work on my RTX 4090 24GB gpu even though the model is only 7GB.