Comment by actualwitch

LLMs had ability to do this from the start, but at some point providers stopped exposing this api to prevent distillation. Just write the prompt, pass the data and sample 1 token returning logits, OSS inference engines can do it easily.