I increasingly feel like "consciousness" is little more than "that thing we humans have... you know what I mean... you have it too!" and that any attempt to elevate it to a more general principle is doomed to failure. Sure, we can easily extend it to other living beings because we share a rich evolutionary history, but beyond that...
As far as AI goes, I think that whatever it is that they have is analogous in significant ways, but also irreconcilably different in significant ways. They aren't proceeding in a straight line towards consciousness either, because why would they? They're doing their own thing which happens to produce similar outputs. It's a problem in terms of value: we value consciousness, because we value ourselves, because we value our output. If it turns out consciousness as we know it is just a minor implementation detail with no cosmic or long term significance, then irrespective of whether AI is conscious or not, we still have to grasp with the challenge of explaining why consciousness is valuable in the first place if not for its outputs.
I'd argue that the reason why we value life, consciousness, the ability to suffer, was always arbitrary, and it relied a lot on the unicity of human experience. It was never robust to the breaking down of human capabilities into inspectable, reusable, upgradable and resettable modules. Or the radical decoupling of capabilities we hitherto considered correlated, like writing about love and feeling love, and many other things.
As far as AI goes, I think that whatever it is that they have is analogous in significant ways, but also irreconcilably different in significant ways. They aren't proceeding in a straight line towards consciousness either, because why would they? They're doing their own thing which happens to produce similar outputs. It's a problem in terms of value: we value consciousness, because we value ourselves, because we value our output. If it turns out consciousness as we know it is just a minor implementation detail with no cosmic or long term significance, then irrespective of whether AI is conscious or not, we still have to grasp with the challenge of explaining why consciousness is valuable in the first place if not for its outputs.
I'd argue that the reason why we value life, consciousness, the ability to suffer, was always arbitrary, and it relied a lot on the unicity of human experience. It was never robust to the breaking down of human capabilities into inspectable, reusable, upgradable and resettable modules. Or the radical decoupling of capabilities we hitherto considered correlated, like writing about love and feeling love, and many other things.