Yeah, it's a really well defined problem space with quantifiable outcomes, so LLMs are very well suited to it.
For lopsy, the biggest issue I ran into was combinatorial bugs: certain sequences of tools and actions would cause a bug to occur that wasn't obvious from isolated tool testing. I solved it by having Claude generate complete projects, full documents with novel concepts chosen randomly, built in the browser with playwright. This continues to expose so much more than individual unit/tool tests, and I have an automated loop that addresses them. (And a bonus from that: I use the outputs to make tutorials to show what the tool can do).
For lopsy, the biggest issue I ran into was combinatorial bugs: certain sequences of tools and actions would cause a bug to occur that wasn't obvious from isolated tool testing. I solved it by having Claude generate complete projects, full documents with novel concepts chosen randomly, built in the browser with playwright. This continues to expose so much more than individual unit/tool tests, and I have an automated loop that addresses them. (And a bonus from that: I use the outputs to make tutorials to show what the tool can do).