Interesting take. You appear to be defining reproducibility to be something like “could I write this source code again myself”. But no one I know uses the term reproducible in the way you’re saying. Everyone I know, including myself, takes reproducibility in this context to mean “if given source code, I can produce a binary /executable that is identical to the one that is produced by some other party when built the same way”
Now I get that “Identical” is a bit more nebulous when it comes to LLMs due to their inherent nondeterminism, but let’s take it to mean the executable itself, not the results produced by the executable.
Now I get that “Identical” is a bit more nebulous when it comes to LLMs due to their inherent nondeterminism, but let’s take it to mean the executable itself, not the results produced by the executable.