Consumer grade GPUs like NVidia's 3090 and 4090 max out at 24 GB VRAM, and those cost $1000-2000 each. You can get higher VRAM but need enterprise GPUs which are in the five figures, easily starting at $30K a pop.
Per this calculator, for training, only gpt2-large and gpt2-medium would work with those two top-of-the-line GPUs.
For inference it's certainly a bit better, only the Llama-2-70b-hf and Llama-2-13b-hf don't fit in that much VRAM, all the other models do.
Per this calculator, for training, only gpt2-large and gpt2-medium would work with those two top-of-the-line GPUs.
For inference it's certainly a bit better, only the Llama-2-70b-hf and Llama-2-13b-hf don't fit in that much VRAM, all the other models do.