Comment by tetete

With 8 bit training you can do ~13B pram LLM on 3090/4090. https://huggingface.co/blog/trl-peft

But it is pretty cheap to rent something at vast.ai or whatever to get 40GB for a final run.