"confabulation" could be "solved" when LLMs realize they are uncertain on a reply and making things up. But that requires the LLM saying "I don't know" and rewarding that more than a wrong guess. That requires a change in loss functions and not even sure if all users desire that.
It is a feature, not a bug.
"confabulation" could be "solved" when LLMs realize they are uncertain on a reply and making things up. But that requires the LLM saying "I don't know" and rewarding that more than a wrong guess. That requires a change in loss functions and not even sure if all users desire that.