Comment by masafej536

They often do fail, at the client level you can just feed the schema validation error message back into the LLM and it corrects itself most of the time. If not the LLM throws itself into a loop until its caller times it out and it sends an error message back to the user.

At the server level it's just a good old JSON API at this point, and the server would send the usual error message it would send out to anyone.