I think you are getting at the need for tiered layers of abstraction and constraint. Simultaneously considering all possible ways to solve a problem doesn't work for humans or the LLMs derived from our use of language. The repeated use of Domain Specific Languages (DSL) in the context of a general purpose programming language gets at this same need to constrain solution spaces within a reasonable boundary.
Once we have quantum LLMs, the need for intermediate abstraction layers might change, but that's very [insert magic here].
Once we have quantum LLMs, the need for intermediate abstraction layers might change, but that's very [insert magic here].