A few months ago there was an MIT talk by Sebastien Bubeck about early tests with GPT-4.[0] He mentions that the tests he ran were on the model before it was RLHFed and if you tried to replicate the tests with the public version the performance was greatly degraded.
[0] https://www.youtube.com/watch?v=qbIk7-JPB2c