Certhas

You are severely misusing the term monopoly.

There is demand for AI that is not US or China controlled. Mistral is serving that demand.

Mathematics has enough cultural capital within the AI companies that they get to have this "advisory board".

Nobody else is getting that. When/If the severe impacts on labor materialize, workers won't get an advisory board. They will just get fired, and the companies will celebrate it as efficiency wins.

I think it's time to get very very real about regulation/taxation. Token/Compute sales tax that gets redistributed as UBI?

Are you talking about the posted article? I agree that it was a fun read, but it did not present any mathematics at all. Like, none. Its not that the author not being an expert made it easy to understand, it's that there was nothing mathematical to understand presented.

If I repeatedly call an LLM in a loop with a markdown document it can edit, would that make it qualify for you?

If I give an LLM to compact its context window, so the context it carries can evolve iteratively over time as more and more things come in, is that enough?

Compacting the context is really a very, very interesting example here. The "next token predictor" is telling an external tool to change all "previous" tokens. So an LLM + a harness that allows compacting the context is no longer just a token predictor at all!

You don't need continuous learning to get interesting dynamics. You just need feedback loops.

Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowledge the difference between a 2500 Neuron fruit fly brains and a human brain.

Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.

Good science, properly digested and presented takes time.

The idea that anything other than a breathless blog post about the latest model snapshot is useless is really poisonous to proper debate on AI issues

Anthropomorphizing is problematic because a human mind is a very bad model for what LLMs are.

A lawnmower is a much much much worse model.

I have never come across this position argued seriously. I would also consider it absurd, as then the question whether we have consciousness depends on unknown properties of fundamental physics (which is not incompatible with determinism, see e.g. Bohms theory). It would therefore be unknown whether humans are conscious. That is at the very least a notion of consciousness that is utterly distinct from any established meaning of the word.

I am a physicist. I worked (briefly) on foundations of quantum mechanics. I discussed compatibilism extensively with philosophers.

I have _never_ come across the position you seem to take here, that determinism has bearing on the question if we are conscious and sentient.

Nonsense. There was one proposal for relevant quantum effects in brain dynamics, and that turned out to be not relevant. Even if they were, you could substitute all quantum randomness with pseudo randomness and obtain an absolutely indistinguishable object.

But even if this were a debate, its absolutely absurd to claim that the question of determinism in the brain has any bearing on our moral standing. If we discover tomorrow that quantum collapse is deterministic and can be derived from an underlying theory, and thus all of physics is deterministic in the good old fashioned Newtonian sense, this would not affect our moral standing in the least.

Your car analogy is very, very confused. If a simulation of a car can get you from A to B, requires the same steering, fuel and servicing, gives you the same tactile feedback, is it a car?

Of course "I care about humans because I am human" is a self-consistent position to take. But now you need to decide if you want to consider a full simulation that faithfully reproduces everything that physically happens between our ears as human. After all I might very well implement this simulation using a bunch of tensor multiplications in an autoregressive setup...

Videos and LLMs are not deterministic in the same sense at all.

LLMs are deterministic in the same sense as biological processes. And a faithful simulation of a brain would have all the properties you note.

I generally agree about the problem with anthropomorphizing. But I don't think Anthropic are doing that. They explicitly write "in biological entities this would be considered a sign of consciousness, but we don't know how to interpret it here".

However, I disagree with your point that "it's an autoregressive function, thus it doesn't matter". Let me explain why:

Assume I do a complete neurological scan of a brain. I then implement this scan in a simulation and run it. Assume that my scan and my simulation of the biology of the brain (and the sensory and motorical inputs and outputs) is good enough that you can now have conversations with the simulation, and in all aspects, this simulation behaves exactly like you expect a human to behave.

Of course this is deterministic. If you take the state of the brain and then run it again, replaying the inputs, you get the exactly same behavior again.

I would argue that the experiences of this simulation are of the same onthological status as our own.

Now I work in dynamical systems. The autoregressive process of LLMs (hooked up to a harness providing it with inputs and outputs) is roughly in the same complexity class I would expect for a brain simulation. A physical simulation of an ODE is also an autoregressive process. The major major difference here is the existence of a latent brain state. But conversely the autoregression on sequences of hundreds of thousands of tokens is a much higher dimensional state than I expect for the latent brain state. In my view this is more an artifact of our inefficient LLM architectures, than a fundamental difference.

Now to be absolutely clear: I don't see evidence that would clearly suggest that LLMs have experiences on the same onthological status as we do. I simply believe this is a reasonable and relevant question to ask.

I like it, and it points in the right direction, but is not directly true: The markers are about interactions, how biological organisms behave in certain test situations.

But it speaks to the central question: Are the tests adequate? Or are they measuring some proxy of what we really care about, and LLMs are merely imitating consciousness.