Comment by jeeeb

I don’t think this is a very good use case. You could do it better with a strong LLM and structured outputs.

The problem is that you want the model to carefully reason about the goal and code.

Zero shot classification with an approach like this isn’t going to do that. It’ll answer on first pass vibes.