Comment by worstspotgain

I'd say the first step is to rule out LoRA. If LoRA is an option, it buys you more than just the rig savings:

- You can deploy multiple specialized LoRAs for different tasks

- It massively reduces your train-test latency

- You get upstream LLM updates for "free," maybe you can even add the training to your CI