Comment by FlyingLawnmower

I spent a couple years building a high performance, expressive library for structured outputs in LLMs. Our library is used by OpenAI for structured outputs on the hosted API. Happy to answer questions on how this works:

User friendly library that connects to lots of OSS model serving backends: https://github.com/guidance-ai/guidance/

Core Rust library written for high performance mask computation (written mostly by my collaborator @mmoskal): http://github.com/guidance-ai/llguidance