Comment by monocasa

Transformers are plenty deterministic. We just decide to run an rng against their output to get the final token in most cases.