whycombilater

For the record, I'm a materialist with a physics degree. I'm not arguing humans are special. All animals are meat machines.

Human input data is sensory data about the world, which they collect in a way that is directed and attention driven. It involves agency. That sensory data is abstracted, by perception, by memory, and influenced by emotional context and the context of prior memory. Humans then record that in the form of language which is a further abstraction. But each abstraction is a label which is in some chain that can be grounded to subjective experience.

That's not true of an LLM. A LLM is a relation model of tokens (which we recognise as labels to things or concepts), based on harvesting the complex relationships between tokens from a massive corpus of labels - which are taken from (amongst other things) the sum total of human expression of the human experience.

So an LLM can tell me about what it feels like to experience the sun on its skin on the beach, but what it is really doing is regurgitating some sample of human expression about what that is. It can regurgitate a story that lies on the surface of recorded human expression, but it's still a regurgitation. It's just processing tokens.

The map is not the territory.

An LLM can find you a novel new route on the map, but it can't explore new territory, it can just tell you about someone else's map.

"Daisy, daisy, give me your answer do..."

It's not about the mechanisms, it's about what it is.

Humans have subjective experiences, they create individual memories, they relate these using language.

LLMs are trained on accounts of the experiences of others to regurgitate a facimilie of that experience.

As a human, my input data isn't a diary account or a book or a home video. It's a real world experience that I participate in, and that I use language to communicate.

An LLM is a facimilie of everything it's made up of. It doesn't have its own agency, it doesn't have its own thoughts. I can ask it what it is like to ride a roller coaster and it will give me some text, but what is that text? It's something that someone else wrote about something the llm has no real experience of.

The llm is in platos cave staring at other peoples shadows.

Nuclear weapons aren't actually that useful unless you're going for a nuclear winter.

You don't have to eradicate them all.

Wait for the robot workforce to be deployed, then visit a few deadly pathogen storage facilities. Society collapses and then you send in the murder bots, or you just ignore the ants.

Humanity has wiped out lots of species without deliberate organised effort.

"If we get a benevolent head of a political system from this, it will be because ... the model broke containment and assumed total control against their will."

Yes, that's what I'm assuming...

The LLM is glorified data product.

Humans may not have many original thoughts, but the thoughts that they arrive at are the product of (at least partially) independent processes.

An LLM is constructed. It no more had its own thoughts than a physical book has written its own story.

We're just going one step back. Instead of picking up a book, we're picking up a book generator. A book generator that was created from scanning in a bunch of books. All those books have authors and the book generator has essentially been authored by that collection of authors.

The book generator can very convincingly tell me what it is like to be a parent, in every way, but all it's doing is regurgitating data. It's never been a parent, it will never be a parent, it has never held a child or even experienced a single emotion. The word "parent" is just a collection of tokens. It's not a label that it uses to make sense of its experience, or other llms experience, of reality.

It's just an overly elaborate chinese room.

Would it be hard?

Based on our current trajectory all it seems to require to me is a basic plan and a bit of patience...

Just how clever can an infinite recursion of someone disappearing up their own backside be?

My subjective experience is mine, driven by me, and I record my thoughts about it on paper.

An LLMs experience is everyone else's subjective view of their history. It doesn't make its own and it never can.

An LLM is a clone of Guy Pearce trapped in a context window which is just reruns of memento with script variations.

I kind of viewed the options as giving a shit, or not giving a shit.

If the AI doesn't give a shit I assumed it would just get rid of us, and if it did give a shit it would be making maximal utility judgements.

Been a lot of things.

The older I get, increasingly, the thing I crave most is real connection.

You could play it around those people for 100 years and it would make no difference.

I imagine they'd also conclude Picard is a loser.

From their own subjective experience.

The dogs don't run the kennels.

If an AGI seizes power I'm not sure what my trust has to do with anything...