Comment by faabian

Language models factor the joint probability p(y, x) as p(y, x) = p(y|x) p(x) which is exact. I.e. if you train a language model on your distribution and sample with temperature 1, you will get the exact same distribution out. If you sample at lower temperature or even greedily, evidently, you will get other distributions.