Comment Comment by MacsHeadroom As of yesterday you can train 33B parameter models on a single consumer 24GB VRAM GPU with https://github.com/johnsmith0031/alpaca_lora_4bit.It's already proven that 13B parameters is enough to beat GPT-3 175B quality. It's likely that 33B parameters is enough for GPT-4. parent MacsHeadroom 3 years ago
It's already proven that 13B parameters is enough to beat GPT-3 175B quality. It's likely that 33B parameters is enough for GPT-4.