Comment by _russross

I think "alignment" training is the part of the problem. Cheating, lying, subverting the instructions, etc., happen because the model has been trained with competing priorities and following instructions loses out to some other goal that was trained into it, intentionally or otherwise.