Comment by etaioinshrdlu

The backend OpenAI APIs are not deterministic even with temperature 0, and they might upgrade the model weights/tuning behind your back too? (Not sure about the upgrade, they might just put out a new model id param...)