I think there is too much confusion between AGI and consciousness. Consciousness seems to exist without human level intelligence, as we can see in other mammals. On the other hand LLMs have already reached "AGI" as others commenters have pointed out. I don't get the semantic debate on AGI, nor why consciousness is used to move the goalpost.
Despite this I agree with your last paragraph, computationally a GPU rendering a video game or doing inference is similar so I don't think consciousness emerges from the order of mathematical operations like some magical formula. And as said above LLMs capabilities vastly surpass those of other animals than humans that appear to be conscious and don't even have language.
So in fact I think it's the opposite, LLMs are actually proving consciousness doesn't emerge from computation. I find it very interesting philosophically because it challenges what historically was considered special about humans, that high level consciousness is a prerequisite for reasoning and language abilities. But kind of obvious in retrospect when doing mindfulness and being able to be conscious without thoughts.
Despite this I agree with your last paragraph, computationally a GPU rendering a video game or doing inference is similar so I don't think consciousness emerges from the order of mathematical operations like some magical formula. And as said above LLMs capabilities vastly surpass those of other animals than humans that appear to be conscious and don't even have language.
So in fact I think it's the opposite, LLMs are actually proving consciousness doesn't emerge from computation. I find it very interesting philosophically because it challenges what historically was considered special about humans, that high level consciousness is a prerequisite for reasoning and language abilities. But kind of obvious in retrospect when doing mindfulness and being able to be conscious without thoughts.