It seems like this problem (differences in how humans and LLMs use probabilistic language) and hallucination are one in the same. LLMs don’t have access to information about how confident they are, so they always choose the most likely response, even if the most likely response isn’t actually that likely. Whereas if a human is unconfident, they’ll express that instead of choosing the most likely response.
Of course, LLMs can still speak about probabilities and mimic uncertainty, but that’s likely (heh) coming from their training data on the subject matter, not their actual confidence.
Humans are interesting because they employ a two-phased approach: when we’re learning, we fake confidence (you’d never write “I don’t know” on a test unless you truly had nothing of value to say), but during inference, we communicate our confidence. Some humans suffer from underconfidence or overconfidence, but most just seem to know innately how to do this.
Can anyone who works on LLMs clarify whether my understanding is correct?
Of course, LLMs can still speak about probabilities and mimic uncertainty, but that’s likely (heh) coming from their training data on the subject matter, not their actual confidence.
Humans are interesting because they employ a two-phased approach: when we’re learning, we fake confidence (you’d never write “I don’t know” on a test unless you truly had nothing of value to say), but during inference, we communicate our confidence. Some humans suffer from underconfidence or overconfidence, but most just seem to know innately how to do this.
Can anyone who works on LLMs clarify whether my understanding is correct?