ooloncoloophid

1. John Searle featured prominently because of his accessibility. I tended to present his Chinese Room argument as a criticism of symbolic AI alone, though I believe he considered it a criticism of sub-symbolic approaches as well.

2. Thomas Nagel's classic article "What Is It Like to Be a Bat?" is a good introduction to qualia, which is how we describe direct conscious experience.

3. Wittgenstein would be important in terms of the impossibility of empathising with, or seeing things from the perspective of, other minds that have emerged in different contexts (such as animals). However, I rarely spoke about him because, frankly, I couldn't understand him in the original!

4. David Chalmers. Writes clearly (and coined?) the 'hard problem' of consciousness and why subjective experience appears difficult to reconcile with a purely physical account of the mind.

4.1 Daniel Dennett. The clearest and most influential critic of the idea that consciousness presents a special explanatory problem.

5. Darwin and others. Comparative psychology (the study of animal minds) strongly suggests, by showing that the antecedents of the human mind are present in animals, that we should reduce our bias that human minds are special, ineffable, or somehow atomic.

6. Jerry Fodor. Modularity: the idea that the mind is composed of modules that specialise in certain tasks (e.g. phoneme perception, syntactic analysis, face recognition) and operate largely unconsciously unless they are dependent on one another in some way. This helps us take a computational approach to the mind. It reminds us that much of what we do mentally might be expressible in computational terms.

Two final ideas off the top of my head:

1. The Ship of Theseus: https://en.wikipedia.org/wiki/Ship_of_Theseus

   When (not all) neurons die off and are replaced by new ones doing the same job, our sense of self and identity is somewhat cast into doubt. In a physical system, if consciousness is a magic or non-material entity of some kind, what is happening to it during this process?

2. Integrated Information Theory. A good attempt to tackle consciousness rationally. The idea is that consciousness corresponds to, or is associated with, the degree to which information in a system is integrated into a unified whole.

Comment · 5 months ago

On Asimov, much as I loved those stories as a teenager, it was only later that I realised the robot stories are largely explorations of how apparently sensible rule-based systems generate unexpected and sometimes harmful outcomes. The Three Laws are presented as hard guardrails, but the stories focus on ambiguities, loopholes, and unintended consequences. I'm not sure how intentional this was; he might have been attempting to make a point about rule-based systems, or perhaps he was following his instinct for drama.

https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

Comment · 5 months ago

I haven’t come across one, unfortunately. You’ll forgive the irony if I suggest talking to a frontier model like Claude about these issues; they are quite accurate, although they’ve been seeded with biases that make them lean in Chiang’s direction. Otherwise, the Stanford Encyclopedia of Philosophy is excellent as a starting point:

https://plato.stanford.edu/

I’ll mention some more sources in my reply above.

Comment · 5 months ago

My background is in cognitive science and psycholinguistics. I spent more than ten years talking to first year psychology undergraduates about whether AIs could be conscious; also did some research on (extremely tiny) AIs in modelling language behaviours.

There is a great deal of good thinking on Chiang's topic by professional philosophers, and there's much to be said for reading them. I won't rehearse their ideas here. Chiang's arguments might be correct; but I suspect they probably aren't, and his error may well stem from characterising human thought as something in its own class, which is probably a cognitive bias that humans have. He might also - I'm speculating - be arriving at his conclusion based on his feelings, which the final paragraph suggests (the comment about the models being based on morally dubious actions).

Speculation aside, we are not, I believe, in a position to make points like he does with any certainty.

Comment · 6 months ago

The point about confidence intervals is a good one and I'd like to see it more often. My neighbour Alan is a good farmer, but I am not.

Comment · 8 months ago

His book 'On Liberty' is the subject of a recent In Our Time episode (BBC Radio Four series on the history of ideas) [https://www.bbc.co.uk/programmes/m002pqnc]. They discuss his childhood and his (apparently very warm) relationship with his father. (Sidenote: first proper In Our Time episode with the new host; he seems fine, but I miss Melvyn Bragg.)

Comment · 1 year ago

I'm half way through this article. The word 'introspection' might be better replaced with 'prior internal state'. However, it's made me think about the qualities that human introspection might have; it seems ours might be more grounded in lived experience (thus autobiographical memory is activated), identity, and so on. We might need to wait for embodied AIs before these become a component of AI 'introspection'. Also: this reminds me of Penfield's work back in the day, where live human brains were electrically stimulated to produce intense reliving/recollection experiences. [https://en.wikipedia.org/wiki/Wilder_Penfield]

Comment · 1 year ago

Beyond consumer-producer relationships, there are many instances where an individual is required to deal with a baroque interface, as I just did when starting to look after an ill parent and figure out what care they could get from the local and state governments; there are forms, definitions to get one's head around, high stakes (get it wrong and you could be breaking the law), and so on. An AI in this case was incredibly helpful, particularly when I was overloaded cognitively and emotionally. There is no particular incentive on the other end of the citizen-government relationship for the government to obfuscate things, but things are sometimes very complicated and provided in verbose language. For those interactions, for that asymmetry, an AI will be very useful.