Comment

Comment by NitpickLawyer

Constrained generation guarantees syntax. It does not guarantee semantic correctness tho. Imagine you want a json object with "hp" and "damage". If you use a grammar, the model will be forced to output a json object with those two values. But it's not guaranteed to get sensible values.

With a 2nd pass you basically "condition" it on the text right above, hoping to get better semantic understanding.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton