Good article. We really need more nuance in the discussion of using LLMs for SE. Not all tasks are the same; we shouldn't expect to use the same process across them!
In my experience, the more constrained the LLM is, either by existing patterns in the code base or by prior art in general, the better the LLM is. When I've asked it do architectural or design work, it has been much less successful. That's one dimension on which LLM capabilities vary. There are certainly more!
In my experience, the more constrained the LLM is, either by existing patterns in the code base or by prior art in general, the better the LLM is. When I've asked it do architectural or design work, it has been much less successful. That's one dimension on which LLM capabilities vary. There are certainly more!