Can anyone explain (or link to an article which explains) why telling Claude things like not to cheat or that there is a holdout set makes the output better? People say that an LLM is just a statistical next token predictor. I've read the article by Stephen Wolfram [0] on this, which makes sense to me, but then I can't see how 'more complex' behavior like cheating or not cheating can arise.
[0] : https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...