Well, imagine advanced AI beyond imagination. It is so advanced that it mimicks all of the human activity in exactly the same way. By definition it would be able to do whatever we are doing (and probably better)
Unless you don't believe that such an AI is possible, in which case I wonder if you have tried GPT-4 and whether you remember what chat bots were like 5 years ago!
This of course would be AGI. It would still not be conscious though, but a lot of people would fall victim to it and believe that it is. And that's what will end us. We will not pass the mirror test (yep, this one https://en.wikipedia.org/wiki/Mirror_test)
Unless you don't believe that such an AI is possible, in which case I wonder if you have tried GPT-4 and whether you remember what chat bots were like 5 years ago!
This of course would be AGI. It would still not be conscious though, but a lot of people would fall victim to it and believe that it is. And that's what will end us. We will not pass the mirror test (yep, this one https://en.wikipedia.org/wiki/Mirror_test)