This basically Searle's Chinese Room argument. It's got a respectable history (... Searle's personal ethics aside) but it's not something that has produced any kind of consensus among philosophers. Note that it would apply to any AI instantiated as a Turing machine and to a simulation of human brain at an arbitrary level of detail as well.
There is a section on the Chinese Room argument in the book.
(I personally am skeptical that LLMs have any conscious experience. I just don't think it's a ridiculous question.)
There is a section on the Chinese Room argument in the book.
(I personally am skeptical that LLMs have any conscious experience. I just don't think it's a ridiculous question.)