Comment by sgtwompwomp

Yup, this is a case where you always want an agent to do that step. So in the prompt you just say “do a focused_action to select the search result with John”, and then the pathfinder agent will cache in it’s memory to delegate that step to a mini computer use agent, just for that particular task.

After the focused action is done, it’ll go right back to deterministic!