Comment by stygiansonic

A simplified explanation, which I think I heard from Karpathy, is that transformer models only do computation when they generate (decode) a token. So generating more tokens (using CoT) gives the model more time to “think”.

Obviously this doesn’t capture all the nuance.