I agree with basically everything said here, BUT i wish people would stop (mis)using the word "deterministic" in this way:
> Compilers are, for the most part, deterministic in a way that current AI tools are not. Given a high-level programming language construct such as a for loop or if statement, you can, with reasonable certainty, say what the generated assembly will look like for a given computer architecture (at least pre-optimization).
> The same cannot be said for an LLM-based solution to a particular prompt.
This is the correct and meaningful difference to point out here, but it has nothing to do with determinism. LLMs could be perfectly deterministic and still suffer from the same problem. The problem isn't that LLMs themselves are nondeterministic, it's that language is imprecise, and language models themselves are (for the most part) black box text processors. An imagined piece of functionality ("feature") has to pass through both of these somewhat opaque steps before it ends up as code, unlike code that is processed by a compiler, where both the constraints and the structured /formal understanding of the input are much stronger which allows us to reason about and trace the relationship between inputs and outputs in ways that we can't for LLMs and natural language.
> Compilers are, for the most part, deterministic in a way that current AI tools are not. Given a high-level programming language construct such as a for loop or if statement, you can, with reasonable certainty, say what the generated assembly will look like for a given computer architecture (at least pre-optimization).
> The same cannot be said for an LLM-based solution to a particular prompt.
This is the correct and meaningful difference to point out here, but it has nothing to do with determinism. LLMs could be perfectly deterministic and still suffer from the same problem. The problem isn't that LLMs themselves are nondeterministic, it's that language is imprecise, and language models themselves are (for the most part) black box text processors. An imagined piece of functionality ("feature") has to pass through both of these somewhat opaque steps before it ends up as code, unlike code that is processed by a compiler, where both the constraints and the structured /formal understanding of the input are much stronger which allows us to reason about and trace the relationship between inputs and outputs in ways that we can't for LLMs and natural language.