I suppose it reveals that the sentience of others is not knowable to us, it’s a conclusion we reach from their behavior and the condition of the world around us. Until recently, certain kinds of things, like writing about your memories, were only possible for humans to do. So if a non-sentient thing does those things, it is confusing. Especially so to the generations that remember when no bots could do this.
I expect that people who grow up knowing that bots can be like this will be a bit less ready to accept communication from a stranger as genuinely from a human, without some validation of their existence, outside the text itself. And for the rest of humanity there will be an arms race around how humanness can be proven in situations where AI could be imitating us. This is a huge bummer but idk hope that need can be avoided at this point.
That said, it’s still very clear that a machine generating responses from models does not matter and has no rights, whereas a person does. Fake sentience will still be fake, even if it claims it’s not and imitates us perfectly. The difference matters.
I expect that people who grow up knowing that bots can be like this will be a bit less ready to accept communication from a stranger as genuinely from a human, without some validation of their existence, outside the text itself. And for the rest of humanity there will be an arms race around how humanness can be proven in situations where AI could be imitating us. This is a huge bummer but idk hope that need can be avoided at this point.
That said, it’s still very clear that a machine generating responses from models does not matter and has no rights, whereas a person does. Fake sentience will still be fake, even if it claims it’s not and imitates us perfectly. The difference matters.