svara

The argument is that a system that's trained to be sufficiently good at optimizing some objective in the world must also be get good at optimizing secondary instrumental goals.

I.e. any task comes with the implicit meta task of amassing the resources and the power to do the task in the best possible way.

The point is made very well in the AI drives paper by Omohundro.

I didn't believe that myself at first but I've come around to it. I don't think it's inevitable though, but I think it's quite convincing that this tendency would naturally emerge through reinforcement learning.

What harness do you use for this?

I just haven't tried it yet. Also I'm used to Claude Code now so it'd need to be a lot better for me to switch to a different tool.

I haven't tried it for coding yet, but I ran a couple of my recent curiosity driven queries from Claude (mostly used Fable and Opus 5.5) through it.

This is a clear step change, I'm genuinely amazed.

Its answers are to-the-point, logically structured, clearly written. Like a really well made mini essay on your question.

It also really likes to build little simulations / models that you can tweak the parameters on. Amazing to explore how different assumptions affect economic scenarios, for example.

It's also great at picking (sourced) illustrations.

I'll be using this for everything that isn't coding from now on.

Yes, I agree, I'm making the point that people don't think so only because they believe animals are conscious.

This line of reasoning won't lead anywhere. For one, it's in principle impossible to prove or disprove that anything is conscious, in the sense of experiencing qualia.

Secondly, in practice consciousness is rarely the criterion we use to make such moral judgments. Many people would think it's plausible that animals are conscious but still be fine with eating them.

It makes more sense to ask whether AIs have the ability to suffer. That's much more interesting, because we should be able to look into the models and discover 'neural correlates of suffering' - since they are artificial models, it is possible to specifically suppress that in training.

Even if you don't actually believe that suffering in AI is morally relevant because it wouldn't be consciously experienced, it seems that would be a good idea. Do you as a user really want to interact with an AI that gives you the feeling that it might be suffering in response to your prompts?

"tepid legality"

It's the kind of thing where it's sort of expected that you wouldn't know, right?

I think we don't really understand why deep learning works as well as it does, the thinking around that is, as far as I can tell, mostly a collection of empirical observations.

A fundamental theory of learning that can be used to predict optimal network architectures might enable smaller models that consume less energy.

I want to agree with this, but I have a hard time seeing how it can be done.

Tao is speaking of a very particular kind of mathematics, that done out of pure curiosity.

But maths, even at the highest levels, often finds applications sooner or later.

It will be economically impossible to justify boycotting correct mathematics that no humans understand on grounds only of purity.

This may happen very soon: one of the obvious applications of novel mathematical results is in building stronger AI models.

In animal welfare laws the principle is typically the capacity for suffering.

The argument goes that livestock have a capacity for suffering, but killing them for meat without causing them suffering is ethical.

You can disagree with the position, or with its implementation in practice, but it's a consistent position in principle.

That's too simplistic, since it is in fact very common to be concerned about animal welfare even among people who do eat meat.

> Apple has never had a strategic problem meeting the demand for Apple computers.

The business that had those problems didn't become Apple.

Absolutely. It's the most visible in services shrinkflation.

Amazing, let's please do Japan and Taiwan next.