voidmain

You are not the farrier in this analogy. You are the horse.

Worse, P(doom|corrigible human replacement level AGI) is also very high, because the dominant strategy in international competition will involve zero humans. So we are screwed even if "alignment" turns out to be surprisingly easy or if there is a plateau right around human level.

It's butlerian jihad or bust, I'm afraid.

The most likely path I see for human survival and flourishing is for AI to do something in the near future so destructive and terrifying that it unites humanity behind permanently banning it at any cost.

It therefore seems like "AI safety" people may actually be working against our safety.

Sort of like how the human economy is limited by the fact that ants can't afford our products and services?

The first thing I do when I see an interesting article title is click on the comments to see if people have noticed that it's AI generated. If I didn't have this option because of the policy you desire, I would probably just stop reading HN entirely.

I'll defend i^4x since I like it better.

(cost x, sint x) is a point on the unit circle x turns counterclockwise from (1,0).

cost x + i sint x is a point in the complex plane x turns counterclockwise from 1.

Now look at integer powers of i, a point in the complex plane 1/4 turn from 1:

i^0 = 1 (0 turns from 1)

i^1 = i (1/4 turn from 1)

i^2 = -1 (2/4 turn from 1)

i^3 = -i (3/4 turn from 1)

and we define complex exponentiation such that, for all real x,

i^x = cost (x/4) + i sint (x/4) (x/4 turn from 1)

How about i^4x? It makes it clearer which direction we are rotating (vs -1 which is a 0.5 turn rotation in either direction) and avoids the garden path confusion that could arise from it not being obvious from the left side of the equation that we are working in the complex plane at all.

> AI writes copy like this because that's how copywriters did it in the training set

I don't think this is true. Someone probably had to write like this, but a base model has to be able to write like anyone. The specific mannerisms of AI appear in post training, either as things that are rewarded or as mode collapse artifacts. ChatGPT didn't get obsessed with goblins because that's incredibly common among humans.

The visualization is of the power set, which is uncountable.

> Q: Do you think KVM vulnerabilities will keep appearing?

> A: Yes. I recommend establishing a sustainable patching process for host hypervisors. Winter is coming.

Vibe coding is doing its part to keep linguistic diversity alive: My wife went to sign up for a doctors appointment, and the form offered choices for "preferred language" that included Old English, ancient Egyptian and Elamite.

I think you have slightly misunderstood the halting problem. There is no computer program H that, for every program P, determines whether P halts. This does not imply that for some particular program Q there is no proof that it halts. In fact in the setting where programs do not take input there is always such a proof (just a trace of the program running to completion). And there are many valid proofs of theorems like "For all x, Q(x) halts" for particular Q. Consider, for example, the case where Q has no control flow at all!

Again, my concern is about things that humans will order robots to do, for fear other humans will do unto them first, and does not depend on a loss of control. I think loss of control is not too unlikely in the process of such an arms race, but it hardly matters.

You don't need superintelligence to get this problem, but you do need human level intelligence across all domains. I don't know how far we are from this, and neither does anyone else.

Governments, initially made of humans, will deliberately create military and industrial robots and assign them to create and scale production chains that go from natural resources to finished war machines. Maybe your government will not do this; in that case it will be helpless against the robot armies of those that do (yes, even if it has a few or few thousand nukes).