Question from someone with a surface-level knowledge of how LLMs work:
As I understand it, LLMs work, at a very high level, by repeatedly choosing the token with the highest probability. Could potential confabulations be flagged by marking low-confidence sequences of text in the output? This seems like it's be particularly good at detecting when the model is making up a name or number, for example.
As I understand it, LLMs work, at a very high level, by repeatedly choosing the token with the highest probability. Could potential confabulations be flagged by marking low-confidence sequences of text in the output? This seems like it's be particularly good at detecting when the model is making up a name or number, for example.