Comment by msl

GP did not suggest anything of the kind, though. They talked about "figuring out where to look" which very different. In that kind of use, it doesn't really matter that the model might hallucinate or draw faulty conclusions, as you're not using it figure out what to do, how or why. The only thing it is telling you is where some information is located, after which you go check it out from the actual source of truth. If the reference turns out to be wrong, you'll notice immediately.