Comment by tatjam

If I was Stephen Wolfram, i'd be screaming (in a very academic way) "you are just talking about computational irreducibility!!" LLMs are very close to computationally irreducible, no chance you can predict its output from the prompt alone. Source code meanwhile is fairly reducible (and is typically reduced to machine code by the compiler). And even complex source code is intentionally designed to be highly reducible via abstractions. Good luck making prompts that reduce the complexity of the LLM behavior!