Comment by mmmehulll

That’s really interesting. The “one or two examples + good error messages” part feels especially important. It suggests the limiting factor may be less finetuning and more whether the model is given a tight representation and a feedback loop it can recover from.