This is why the Turing Test gets talked about a lot (although I gather Turing's original 'imitation game' suggestion had slightly different emphasis).
If you can reliably reproduce a situation where a human can't tell whether they are conversing with an AI or a Human, then thats a kindof yardstick.
So yes we can't define consciousness but we've got a sortof yardstick for when we might consider something to have achieved it.
And people think the Turing Test is too easy but the point is that the 'tester' should ask difficult questions to try and probe the depth of thinking of their subject.
If you can reliably reproduce a situation where a human can't tell whether they are conversing with an AI or a Human, then thats a kindof yardstick.
So yes we can't define consciousness but we've got a sortof yardstick for when we might consider something to have achieved it.
And people think the Turing Test is too easy but the point is that the 'tester' should ask difficult questions to try and probe the depth of thinking of their subject.