Comment by ajmurmann

For every production use case I build a eval suite which I use for prompt tuning and model evaluation and config. How else do you establish your model and prompt combination works? How do you upgrade to a newer model or decide on a fallback?

This same suite can simply be run very handoff to switch to a new prod model.