Temperature = 0 would always return the highest probability token at every generation step and given a single distribution of tokens at each generation steps,that would become deterministic.
Where that gets murky is… perturbation inside the models themselves, such as Mixture of Experts (MoE) models who’s more internal parameter activation is not so deterministic (and therefore the tokens they generate are not deterministic across runs)
Where that gets murky is… perturbation inside the models themselves, such as Mixture of Experts (MoE) models who’s more internal parameter activation is not so deterministic (and therefore the tokens they generate are not deterministic across runs)