> We need to stop thinking of LLMs as purely a piece of software
Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token. Humans see that token and assign it a qualitative but unfalsifiable intelligence score.
The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not. A cat is intelligent regardless of what humans think. Is an LLM intelligent regardless of what humans think?
If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world? If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token. Humans see that token and assign it a qualitative but unfalsifiable intelligence score.
The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not. A cat is intelligent regardless of what humans think. Is an LLM intelligent regardless of what humans think?
If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world? If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).