One of the most powerful ways to integrate LLMs with existing systems is constrained generation. Libraries such as outlines[1] and instructor[2] allow structural specification of the expected outputs as regex patterns, simple types, jsonschema or pydantic models. Llama.cpp supports bnf grammars.
These outputs often consume significantly fewer tokens than chat or text completions.
These outputs often consume significantly fewer tokens than chat or text completions.
[1] https://github.com/outlines-dev/outlines
[2] https://github.com/jxnl/instructor