Comment by slow_typist

Why is it that LLMs are ‘stochastic’, shouldn’t the same input lead to the same output? Is the LLM somehow modifying itself in production? Or is it just flipping bits caused by cosmic radiation?