Comment by renee_sanscarte

There are two issues here though. Firstly, what is the right thing to specify? (This is an ethical proposition, even for fairly simple agents.) Secondly, how can this specification be reliably encoded in the AI's world model? (This gets technically more more challenging the more complex the agent is.)