For all the fears of AGI, these are the more concrete nefarious uses we can actually reason about. It is a point I often make that we don't need AGI for AI to already become very disturbing in its potential use.
The other point, is that technically this AI is not "unaligned". It is doing exactly what is requested of the operator.
The implications are that humanity suffers in either scenario, either by our own agency in control of power we are not prepared to manage or we will be managed by power that we can not control.
The other point, is that technically this AI is not "unaligned". It is doing exactly what is requested of the operator.
The implications are that humanity suffers in either scenario, either by our own agency in control of power we are not prepared to manage or we will be managed by power that we can not control.