Ask HN: When is fine-tuning a small LLM worth it?

I'd be interested in hearing about your experiences. What kind of task did you use it for, what model did you train, and what were the results?

Feel free to share examples of what you've tried.