Comment

Comment by AnimalMuppet

Alignment is not possible for an AGI. Or, at best, it's only provisionally possible.

Consider what alignment means: It's an AGI, but there are certain goals that it cannot choose. If that's the case, then I assert that it is not actually general. If it is general, then it will decide what goals it will pursue, and you can't stop it from doing so.

The best you can do is load it with an initial set of goals (and perhaps values), and hope that it doesn't decide to change them. But you have no way of making sure that it can't change them without making it not a general intelligence.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton