Comment by dwaltrip

You need to account for reward hacking and unexpected outcomes from training.