Comment by dfdydx

Not all transformers are _language_ models - the sequences of tokens don't have to be sequences of words.