But they're not exactly lying. Lying assumes an intent to deceive. It's because we know an LLMs limitations, that it makes sense to ask it the opposite question/the question without context etc.
If it was easy to look up/check the fact without an LLM, wary users probably wouldn't have gone to the LLM in the first place.
If it was easy to look up/check the fact without an LLM, wary users probably wouldn't have gone to the LLM in the first place.