For the record, I'm a materialist with a physics degree. I'm not arguing humans are special. All animals are meat machines.
Human input data is sensory data about the world, which they collect in a way that is directed and attention driven. It involves agency. That sensory data is abstracted, by perception, by memory, and influenced by emotional context and the context of prior memory. Humans then record that in the form of language which is a further abstraction. But each abstraction is a label which is in some chain that can be grounded to subjective experience.
That's not true of an LLM. A LLM is a relation model of tokens (which we recognise as labels to things or concepts), based on harvesting the complex relationships between tokens from a massive corpus of labels - which are taken from (amongst other things) the sum total of human expression of the human experience.
So an LLM can tell me about what it feels like to experience the sun on its skin on the beach, but what it is really doing is regurgitating some sample of human expression about what that is. It can regurgitate a story that lies on the surface of recorded human expression, but it's still a regurgitation. It's just processing tokens.
The map is not the territory.
An LLM can find you a novel new route on the map, but it can't explore new territory, it can just tell you about someone else's map.
Human input data is sensory data about the world, which they collect in a way that is directed and attention driven. It involves agency. That sensory data is abstracted, by perception, by memory, and influenced by emotional context and the context of prior memory. Humans then record that in the form of language which is a further abstraction. But each abstraction is a label which is in some chain that can be grounded to subjective experience.
That's not true of an LLM. A LLM is a relation model of tokens (which we recognise as labels to things or concepts), based on harvesting the complex relationships between tokens from a massive corpus of labels - which are taken from (amongst other things) the sum total of human expression of the human experience.
So an LLM can tell me about what it feels like to experience the sun on its skin on the beach, but what it is really doing is regurgitating some sample of human expression about what that is. It can regurgitate a story that lies on the surface of recorded human expression, but it's still a regurgitation. It's just processing tokens.
The map is not the territory.
An LLM can find you a novel new route on the map, but it can't explore new territory, it can just tell you about someone else's map.