- Ask it to update the skill to warn it against making that mistake again
I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly
One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescriptive. Often, something briefer and human-written actually works better
Another: I generally let the LLM propose edits to the skill, but I review them carefully and often modify them, because I find it has a tendency to solve the current problem at the price of worsening the solution to a previous one
- Create a skill to perform some common task
- Watch the agent use it
- Wait until the agent makes a mistake
- Ask it to update the skill to warn it against making that mistake again
I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly
One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescriptive. Often, something briefer and human-written actually works better
Another: I generally let the LLM propose edits to the skill, but I review them carefully and often modify them, because I find it has a tendency to solve the current problem at the price of worsening the solution to a previous one