Comment by coderintherye

The last paragraph does the heavy-lifting.

Everyone has a different idea of what is permissable. We can't even solve alignment amongst humans, what makes us think it is possible to solve alignment with AIs? It's irreducible complexity.