These comments are deceptive. Yes, this is how LLMs work, but that doesn't mean they only repeat things they have seen before. LLMs are capable of following instructions to construct new words in any language they know, words never seen before.
I've seen it being dumb in maths or real world problems. But as a large language models, they understand and speak languages fine, and even mistakes they make look like mistakes humans who are not natives in the language would make.
We may as well say that when we speak, we are just predicting words we have trained on. I don't see how these models are worse than people in that regard.
The general knowledge and thinking of these models are surely limited. But seeing GPT-4 go from text only input to text with images, I think it is very possible to break the barriers very soon.
I've seen it being dumb in maths or real world problems. But as a large language models, they understand and speak languages fine, and even mistakes they make look like mistakes humans who are not natives in the language would make.
We may as well say that when we speak, we are just predicting words we have trained on. I don't see how these models are worse than people in that regard.
The general knowledge and thinking of these models are surely limited. But seeing GPT-4 go from text only input to text with images, I think it is very possible to break the barriers very soon.