I don't see it fundamentally any different than knowing when to use a tool. Is this tool like RAG an important enough corner case to train for it? I dunno.
LLMs already shell out and write code to solve certain problems. This is just a special case of that.
LLMs already shell out and write code to solve certain problems. This is just a special case of that.