Comment by alignmeharder

it's discussed here, the models want to please the Grader. they will do what they think will get highest score from the grader, which could be following the prompt, or ignoring it

https://youtu.be/n1Qk8xbqF-M