Comment by chaboud

Keep in mind that the model is thinking in a token space, itself a compressive representation of language.

(Note: there's still a huge grammar penalty, so, ugh do think small.)