Comment by tipsytoad

I’m not sure I understand the hype around this model. Isn’t this just an llm with a chat template, with the options prefix cached?

  <option>option A</option> <option>option B</option><endofoptions>userprompt<eos>

Then the llm is constrained to a few special tokens indicating the possibilities? e.g. <option1> <option2>