> isn’t ‘holding a representation of a fact or relation’ basically what we call ‘knowing’ something?
No. Knowledge is fundamentally and inextricably tied to our experience of reality. The billions of years of evolution that created our perceptual framework has created a bridge to understanding reality machines don’t have.
I’d argue a chicken brain has a far better greater amount of true knowledge than any of these language models.
That does not diminish the utility and sophistication of these language models. But they are not knowledge. They are a disconnected meta map of what we have written down.
We calibrate these language models to knowledge because our language usually corresponds to things we know.
If the internet was mostly populated with sentences like “a druid flew into houston, cats pool into mud vampires wealth umpire onion buy now saxophone”, a language model mapping it all would be useless.
No. Knowledge is fundamentally and inextricably tied to our experience of reality. The billions of years of evolution that created our perceptual framework has created a bridge to understanding reality machines don’t have.
I’d argue a chicken brain has a far better greater amount of true knowledge than any of these language models.
That does not diminish the utility and sophistication of these language models. But they are not knowledge. They are a disconnected meta map of what we have written down.
We calibrate these language models to knowledge because our language usually corresponds to things we know.
If the internet was mostly populated with sentences like “a druid flew into houston, cats pool into mud vampires wealth umpire onion buy now saxophone”, a language model mapping it all would be useless.