If you want to run decently heavy models, I'd recommend getting at a minimum getting 48GB. This allows you to run 34b llama models with ease, 70b models quantized, mixtral without problems.
If you want to run most models, get 64GB. This just gives you some more room to work with.
If you want to run anything, get 128GB or more. Unquantized 70b? Check. Goliath 120b? Check.
Note that high end consumer gpus end at 24GB VRAM. I have one 7900xtx for running llms, and the best it can reliably run is 4-bit quantized 34b models, anything larger is partially in regular ram.
If you want to run most models, get 64GB. This just gives you some more room to work with.
If you want to run anything, get 128GB or more. Unquantized 70b? Check. Goliath 120b? Check.
Note that high end consumer gpus end at 24GB VRAM. I have one 7900xtx for running llms, and the best it can reliably run is 4-bit quantized 34b models, anything larger is partially in regular ram.