Something I often think about is how we can barely define what AGI, consciousness, etc are. We may be pretty sure that what we have currently is an illusion, but at which point is the illusion good enough that it no longer matters? Especially with regards to my first question.
It's hard to say it's not X when we can't really define X.
It's hard to say it's not X when we can't really define X.