> which is clearly an overstatement meant to clear up hype, but is untrue
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.