Comment by monocasa

With llama.cpp, you can give it an ebnf grammar and constrain the output to any formal grammar you wish.

And how is the confidence score any different than the softmaxed token probabilities you get out of running every llm?