The problem with this article starts at the very begining. Where is says “The last few years have seen a wave of hysteria about LLMs becoming conscious and then suddenly attempting to kill humanity.”
That is not a fair description of the fears of AGI. The idea is not that LLMs themselves become a threat. It is not like someone will ask Llama one too many times to talk like a pirate and it will climb out of the GPU and strangle humanity.
It is more likely that an AGI will resemble some form of reinforcement learning agent applied by humans on the real world.
There are multiple entities on Earth whose stated goal is to make an AGI. Deep mind, OpenAI, readily comes to mind. There could be others who keep secret about their projects for strategical reasons. (Militaries and secret services of the world.) They can use the success of LLMs to get more funding for their projects, but the AGI need not be otherwise a descendant of LLMs.
This misunderstanding then goes through the whole article. Under the subtitle “Human writing is full of lies that are difficult to disprove theoretically” which only matter if you think that the AGI needs to learn from text. As opposed to conducting its own experiments, or gaining insight into raw sensor data from experiments.
That is not a fair description of the fears of AGI. The idea is not that LLMs themselves become a threat. It is not like someone will ask Llama one too many times to talk like a pirate and it will climb out of the GPU and strangle humanity.
It is more likely that an AGI will resemble some form of reinforcement learning agent applied by humans on the real world.
There are multiple entities on Earth whose stated goal is to make an AGI. Deep mind, OpenAI, readily comes to mind. There could be others who keep secret about their projects for strategical reasons. (Militaries and secret services of the world.) They can use the success of LLMs to get more funding for their projects, but the AGI need not be otherwise a descendant of LLMs.
This misunderstanding then goes through the whole article. Under the subtitle “Human writing is full of lies that are difficult to disprove theoretically” which only matter if you think that the AGI needs to learn from text. As opposed to conducting its own experiments, or gaining insight into raw sensor data from experiments.