Chris_Newton

I’m generally cautious/sceptical about the current LLM hype, but if that did ever happen and it worked to a useful degree, it might truly be a significant advance. I’m not saying that just because of the LLMs themselves, though.

To reach that point, we would first need to overcome the classic problems in this area:

1. How can you specify the required behaviour for a system in precise detail without ending up with a specification language that is essentially another programming language with similar complexity to the ones we use today?

2. How do you solve the sufficiently smart compiler problem so that producing reasonably efficient and reliable implementations of those declarative specifications is not only viable but expected?

And then, if you still need someone (let’s call them a “developer”) to carefully specify the required behaviour of the system in detail, and you also know how to reliably generate a good implementation of a specification, how much value is left for LLMs to add in this picture?

Maybe LLMs really could be trained to reliably produce good implementations in that scenario? Never say never, but it seems more likely to me that we would develop deterministic algorithms for doing so. That’s already a very hard problem that a lot of smart people have explored, but if a solution could be found, most likely it would be strictly better than relying on LLMs in this part of the process.

Maybe there would still be a role for LLMs to convert natural language problem statements into specifications? Maybe for systems that don’t really matter, that would be enough and a big time saver. However, I’m sceptical about how much time it would be likely to save for systems that did need to work, where you would still need careful design and review of all the detailed specifications by developers anyway.

Maybe LLMs would end up in a similar role to the scaffolding tools that are popular in some ecosystems today, where you feed in some basic choices and the tool then produces a first pass at the files and code you might need, which you take as a starting point for the real work but without wasting so much time on boilerplate? That seems realistic and useful, but it’s a relatively small part of what would then be the overall development process.

In short, I would love for the existence of LLMs/agents to push us towards those better programming languages and tool chains, but if that did ever happen, I wonder how much benefit LLMs/agents could still offer once developers had the more powerful languages and tools available and could work at a “higher level of abstraction” themselves.

A simple task, migrating a typical password reset flow as part of updating a long-lived web application from legacy libraries and software architecture to modern equivalents, apparently cost roughly the same as 2 months of Pro subscription, over the equivalent of about half a working day in wall time.

It produced code of decent quality at a small scale, but it wasn’t always on point architecturally. It also had a tendency to drift off topic and try to tangle up other changes it decided should be made with the main change we were supposed to be working towards. So even for a routine task, based on a plan developed using the harness first and with the agents working under close supervision, a near-SOTA model is still producing quality on par with a decent mid-level developer but substandard for anyone senior+ in this case.

Moreover, based on a direct comparison with other migration tasks of similar complexity that I’d already done by hand, it was actually a bit slower overall to work this way. I had to babysit Claude throughout and review everything it proposed carefully, both to avoid subtle errors (it would have made several) and to prevent drifting off track. I also had to spend a significant amount of time cleaning up its final output to an acceptable standard after the session. Those two overheads more than cancelled out the much faster code generation an LLM offers under favourable conditions.

So for now, I remain sceptical about these high multiples of improved productivity that I keep seeing claimed online from people who are apparently writing almost everything using AIs now. I could certainly have achieved a multiple of my normal productivity by YOLOing everything without reviewing it in detail and then accepting the output code without tidying anything up. However, I doubt this codebase would still have been good enough for normal human developers to work on it reasonably after even 10 or 20 AI-led sessions like that. The architecture would have degraded significantly and the test suite would have been large and largely pointless. And again, this wasn’t rocket science in this experiment, it was completely unremarkable maintenance of a relatively small and simple web application.

Hadn’t good developers been focussing on those things, and other high-level modelling that relates the software system back to the underlying real world domain, for many years before the LLMs were in the picture?

I’d even say that it’s one of the most reliable markers of a more senior/experienced developer that their code reads like a clear and logical explanation of what the system does and why, with concerns well separated and minor details and technicalities abstracted away so they don’t clutter the rest of the code.

I agree with the spirit of what you wrote, but my recollection of the timeline is different. The first decade of the 2000s was peak Crazy Agile Advocacy, but IIRC it wasn’t until the 2010s that unit testing really became almost universal practice. Much before that and it was still tangled up with XP, TDD and lots of other things that certainly weren’t universally accepted as good practices (notwithstanding the strident advocacy of a certain group of consultants/authors/speakers/bloggers and their fans).

I remember, back in the mid-2000s, when we had some consultants brought in to talk about different aspects of quality and testing. There were several working groups, each led by one of those external consultants, and one of them was about unit testing. This was in a relatively large software development organisation for the time, a few thousand people, and while some parts of the organisation had some form of automated testing operating by then, it definitely was not the case that the well-known products produced by the organisation all had a unit test suite. Other practices we’d consider routine today, such as peer code reviews, were also in their infancy during that period: some were doing them, many were not, and generally we had much less experience of how to do them effectively than we have today.

As an industry, I don’t think we really matured in how even the most ardent fans of unit testing were writing test suites until the 2010s either. In the 2000s, we still had lots of people mocking the entire universe and then writing unit tests that were 99% testing those mocks because of 100% test coverage requirements, and similar dogmatic nonsense.

By the 2020s, I think there was much more awareness of that automated testing is generally a good idea, but there are different kinds/levels of automated testing and finding a mix that suits each project’s specific needs is important. One of the great benefits from the more recent AI tools, particularly the agentic ones over the past year or so, has been that it has clearly demonstrated both the value of a good automated test strategy and how much of a waste of time vacuous tests are.

Meta-moderation was an interesting idea, though they seemed to have stopped promoting it much by the time I stopped posting there often. I’m not convinced it’s as effective as having a small pool of “super-moderators” who can not only affect the prioritisation/visibility but also comment themselves to guide contributors in positive directions, but practically speaking, it might be more scalable.

On the positive side, all of the above have attracted many people to their communities who have contributed useful or interesting points. We all give away our thoughts and experience for free while participating in these discussions, but we gain in return from the freely shared knowledge and experiences of others. I also appreciate those who take the time to vote/moderate so that the best contributions stand out. Overall I find these online discussions extremely valuable and I’m sure others do as well.

On the negative side, there are some common failure modes. There have always been the trolls who will post offensive or misleading comments, and even when it’s a small minority, they can be disproportionately disruptive. There have always been the Dunning-Kruger contributors who would insist they were correct even as others tried to explain why they weren’t, and then the people who do know what they’re doing feel obliged to waste time repeatedly setting the record straight so no-one comes along later and gets misled by the incorrect or misleading contributions. I will never understand the current fascination with getting AI bots to contribute mediocre or just plain wrong comments in these discussions. But the worst recurring pathology by far, IMHO, is when there is some form of community moderation but that goes off the rails. It killed SO by deterring good contributors for petty reasons. It has killed many a promising subreddit; I have recently given up participating in several myself that used to be interesting, because their moderators started killing entire posts retrospectively, which repeatedly cut off discussions where some contributors had already taken the time to write up good solutions to someone’s problem or share their relevant experiences.

I’m not sure anyone has really got this right at scale yet. On smaller sites like HN, the moderation can be very good, but that relies on the fact that it can be managed by a small number of decent people. If your community is big enough that it needs to be more self-policing then the time-honoured question of quis custodiet ipsos custodes? is as relevant as ever. I strongly suspect that the only real answer to this is some kind of hierarchy where the operators of a forum set culture from the top, then just as a few negative contributors can spoil things for everyone and so some form of moderation is introduced, so a few negative moderators can spoil things for everyone and so some ability to guide or if necessary remove the use of moderation privileges is needed.

Personally, I’m content being a dinosaur who writes one letter at a time (in handwriting that has been praised for its neatness and clarity ever since I was at school myself) or uses computers to render the text for me (where I have long had an interest in typography and quite enjoy making pretty text using elaborate cursive fonts, but for special effects and interest, not for body text and legibility).

For reasons unknown, LinkedIn seems to have decided that I’m not me a few months ago and blocked my account, though it would apparently be willing to reconsider as long as I provide whatever it is that Persona wants these days. (Evidently contacting me directly via my company — where my role as one of the directors is a matter of public record and my email address was listed in my LinkedIn profile — was too much trouble. :sigh:) Since I have no interest in giving any personal information to Persona, I no longer use LinkedIn and remain blissfully ignorant of all the AI-driven content that I keep seeing complaints about, but I do miss the occasional good news stories about people I actually know. I should probably send a formal GDPR request at some point, since my profile is probably quite misleading by now.

Sorry, but I don’t understand what you mean here. What do we win by being faster at producing the wrong things?

I’ve been noticing that this simple reality explains almost all of both the good and the bad that I hear about LLM-based coding tools. Using AI for research or to spin up a quick demo or prototype is using it to help plot a course. A lot of the multi-stage agentic workflows also come down to creating guard rails before doing the main implementation so the AI can’t get too far off track. Most of the success stories I hear seem to be in these areas so far. Meanwhile, probably the most common criticism I see is that an AI that is simply given a prompt to implement some new feature or bug fix for an existing system often misunderstands or makes bad assumptions and ends up repeatedly running into dead ends. It moves fast but without knowing which direction to move in.

I completely agree about the problem of someone deciding to turn these kinds of scanning tools on and then expecting they’ll Just Work. I do think the better tools can provide a lot of value, but they still involve trade-offs and no tool will get everything 100% right, so there will always be a need to review their output and make intelligent decisions about how to use it. Scanning tools that don’t provide a way to persistently mark a certain result as incorrect or to collect multiple instances of the same issue together tend to be particularly painful to work with.

Clearly it is still possible to generate a false positive if, for example, CodeQL’s algorithm thinks it has found a path through the code where unsanitised user data can be used dangerously, but in fact there was a sanitisation step along the way that it didn’t recognise. This is the kind of situation where the theoretical result about not being able to determine whether a semantic property holds in all cases is felt in practical terms.

It still seems much less likely that an algorithm that needs to produce a specific demonstration of the problem it claims to have found will result in a false positive than the kind of naïve algorithms we were discussing before that are based on a generic look-up table of software+version=vulnerability without any attempt to determine whether there is actually a path to exploit that vulnerability in the real code.

I too declined on privacy grounds.