Ok, so if we take AI safety / AI existential risk as real and important, there are two possibilities:
1) The only way to be safe is to cede control to the most powerful models to a small group (highly regulated corporations or governments) that can be careful.
2) There is a way to make AI safe without doing this.
If 1 is true, then... sorry, I know it's not a very palatable solution, and may suck, but if that's all we've got I'll take it.
If 2 is true, great. But it seems less likely than 1, to me.
The important thing is not to unconsciously do some motivated reasoning, and think that AGI existential risk can't be a big deal, because if it is, that would mean that we have to cede control over to a small group of people to prevent disaster, which would suck, so there must be something else going on, like these people just want power.
1) The only way to be safe is to cede control to the most powerful models to a small group (highly regulated corporations or governments) that can be careful.
2) There is a way to make AI safe without doing this.
If 1 is true, then... sorry, I know it's not a very palatable solution, and may suck, but if that's all we've got I'll take it.
If 2 is true, great. But it seems less likely than 1, to me.
The important thing is not to unconsciously do some motivated reasoning, and think that AGI existential risk can't be a big deal, because if it is, that would mean that we have to cede control over to a small group of people to prevent disaster, which would suck, so there must be something else going on, like these people just want power.