LLMs are inherently deterministic. The way everyone deploys LLMs leads to non-deterministic results, but there's nothing† stopping providers from offering deterministic evaluation if they choose.
All the sources of randomness are under the control of the provider, even if today's deployment structures mean providers introduce extra randomness due to the concurrent nature of the evaluation. Serialise the computation, feed it from a pRNG, and you have a fully deterministic result. But providers don't want to offer a deterministic result, and especially not one as fragile, expensive, and inefficient as a full serialisation would be.
†: For variants of "nothing" that include cost and deployment challenges.
All the sources of randomness are under the control of the provider, even if today's deployment structures mean providers introduce extra randomness due to the concurrent nature of the evaluation. Serialise the computation, feed it from a pRNG, and you have a fully deterministic result. But providers don't want to offer a deterministic result, and especially not one as fragile, expensive, and inefficient as a full serialisation would be.
†: For variants of "nothing" that include cost and deployment challenges.