Bit confused what the value add is over a framework like DSPy. This still requires you to create an eval dataset with ground truth, basically the only hard part of using DSPy. Easily getting the optimized prompt and having some metrics out of the box is not worth nearly $1k/mo IMO
Side note: I’ve had a lot of luck combining automatic prompt optimization with finetuning. There is definitely some synergy https://raw.sh/posts/chess_puzzles
Side note: I’ve had a lot of luck combining automatic prompt optimization with finetuning. There is definitely some synergy https://raw.sh/posts/chess_puzzles