Comment by greenail

you can set the "temperature" which is a lever on how stochastic the prediction is. If you are doing your own inference this is clear and easy. If you are consuming tokens this is outsourced.