Comment by jeeceebees

You can just mask the output probabilities for each token based on which options are valid according to a grammar.

There are quite a few open source implementations of this e.g. https://github.com/outlines-dev/outlines