Comment by khimaros

created something similar but specific to llama-server. it takes advantage of logprob output and completes decisions in a single forward pass https://github.com/khimaros/verdict