Comment by kfallah15

Great question. You need the simulator to be realistic enough that the optimizer cannot reward hack, but it does not need to be perfectly realistic.