You can tell with some of the more recent models that they have been fine-tuned in such a way to produce a lot of descriptions that sound impressive but have no real meaning. In terms of production of semantic text, the recent models are significantly worse than a year ago.
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.