What's wrong with "humans/biologists have no evidence how the human language works. AI practicioners have one important piece of evidence: they can demonstrate somewhat-human-like language processing using statistical techniques, therefore they have the best available evidence". Nobody else can demonstrate it.
But of course, this is simply the non-religious version of "humans have a soul, and machines, by definition don't". If necessary people drag quantum physics into that argument ...
The truly forbidden argument is that we don't have any definition of a soul, and in fact plenty of evidence humans don't have a soul, such as large "soul"/character changes with occuring with physical damage to the neocortex.
This also means the discussion is moot: people are now using LLM's to pass the Turing test on a large scale for all sorts of purposes. From scamming people to management (let's assume there's a difference there). From programming to teaching ugly bags of mostly water new languages. The point where people discover their wife/kids/... have more of a relationship with an LLM speaking for them than with them personally is now just a matter of time away.
But of course, this is simply the non-religious version of "humans have a soul, and machines, by definition don't". If necessary people drag quantum physics into that argument ...
The truly forbidden argument is that we don't have any definition of a soul, and in fact plenty of evidence humans don't have a soul, such as large "soul"/character changes with occuring with physical damage to the neocortex.
This also means the discussion is moot: people are now using LLM's to pass the Turing test on a large scale for all sorts of purposes. From scamming people to management (let's assume there's a difference there). From programming to teaching ugly bags of mostly water new languages. The point where people discover their wife/kids/... have more of a relationship with an LLM speaking for them than with them personally is now just a matter of time away.