So, this is actually an aspect of superintelligence that makes it way more dangerous than most people think. That we have no way to know if any given alignment technique works for the N+1 generation of AIs.
It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created.
It cuts down our ability to react, whenever the first superintelligence is created, if we can only start solving the problem after it's already created.