Human beings act in accordance with their genetics and life experiences. They do this to such a degree that people can, at a young age, internalize maladaptive behaviors which require an enormous level of self-reflection to overcome, if they even can.
I think presuming that humans don't act "as instructed" is a bit too generous an assumption.
LLMs are capable of declining to do things. They're also capable of trying to do things and getting them subtly (or very) wrong.
A lot of this hinges on defining consciousness. I think there is more to the story than what human beings experience.
Do LLMs have an experience very similar to human beings that we would describe as consciousness? No, I don't think that.
I think the bar for consciousness is something more like experiencing a dream. You have a "mind's eye". The experience of existing, rather than nothing. Does that exist somewhere under an LLM? I don't know, but I think confidently claiming that the answer is no seems a bit of a rush judgment.
I think presuming that humans don't act "as instructed" is a bit too generous an assumption.
LLMs are capable of declining to do things. They're also capable of trying to do things and getting them subtly (or very) wrong.
A lot of this hinges on defining consciousness. I think there is more to the story than what human beings experience.
Do LLMs have an experience very similar to human beings that we would describe as consciousness? No, I don't think that.
I think the bar for consciousness is something more like experiencing a dream. You have a "mind's eye". The experience of existing, rather than nothing. Does that exist somewhere under an LLM? I don't know, but I think confidently claiming that the answer is no seems a bit of a rush judgment.