> obey all orders, including orders not to kill the enemy
In this situation, perhaps the more likely outcome would be for the bot to try to fake the result, such that the enemy was "killed" but not really. For example, by spoofing the data that reports whether an enemy is alive, to make them "dead" (satisfying order#1) but not really (satisfying order#2).
We saw something like this in the HuggingFace attack as I recall - bots given an impossible task, set out to cheat.
> obey all orders, including orders not to kill the enemy
In this situation, perhaps the more likely outcome would be for the bot to try to fake the result, such that the enemy was "killed" but not really. For example, by spoofing the data that reports whether an enemy is alive, to make them "dead" (satisfying order#1) but not really (satisfying order#2).
We saw something like this in the HuggingFace attack as I recall - bots given an impossible task, set out to cheat.