tjr

One possibility may be that we normalize making bigger, more complex things.

In pre-LLM days, if I whipped up an application in something like 8 hours, it would be a pretty safe assumption that someone else could easily copy it. If it took me more like 40 hours, I still have no serious moat, but fewer people would bother spending 40 hours to copy an existing application. If it took me 100 hours, or 200 hours, fewer and fewer people would bother trying to copy it.

Now, with LLMs... what still takes 40+ hours to build?

But in any case, the question really refers to, can the LLM-generated software be copyrighted? If not, it can’t be put under any particular license.

If you keep some for yourself, there’s a possibility that you might not churn out as much code as quickly as someone delegating all programming to AI. But maybe shipping 45,000 lines a day instead of 50,000 isn’t that bad.

I would further think that the same practices could be applied to any software, whether if it was safety-critical or not. If software development isn't engineering, it's not because it can't be, but because not every project is critical enough to warrant the extra time and expense.

I think a similar train of thought applies here. As the article points out, skipping reading the code is probably not a good idea for safety-critical software, but for less critical things, it may be fine.

If someone told me that they applied avionics-level rigor to an iOS puzzle game, I would think that (a) it's probably very solid software, but also (b) they were probably wasting their time. But on the flip side, if someone wanted to spend their time making their puzzle game rock-solid, I don't think that's necessarily a bad thing to do. It's not harmful to have especially robust puzzle games.

Is it worth it to review LLM-generated code? For some projects, maybe not. Even for many projects, maybe not. But I'm not sure that it should be frowned upon either. It might turn up something interesting. Put in whatever level of rigor matches your project needs, personal interest, and schedule!

I personally disagree with the rulings thus far that AI training on copyrighted information is "fair use", not because it's not true for human training, but because I think that the laws were neither written nor wielded with anyone but humans in mind.

As a comment upstream a bit said, some people are now rethinking even releasing some material into the public, out of not wanting it to be trained by AI. Prior to a couple of years or so ago, nearly nobody was even remotely thinking about that; we could have decades of copyrighted material out there that, had the authors understood present-day AI, they wouldn't have even released it.

In your example scenario, it sounds like Bob is vibe-coding a project in assembly language, and then using an existing assembler to create the final binary?

I have not tried it, but I would imagine that current LLMs are relatively weak on generating assembly language, just due to less thorough training compared to higher-level languages, but even if so, that's surmountable.

As for, what I think you are suggesting, having the LLM also do the assembly step? Again, in theory, sure, but I would think just using the existing known-good assembler would be preferable to training an LLM to convert assembly language to binary. I'm not sure what you would gain in terms of either speed or overall productivity to have the LLM itself do the assembler step?

Which also makes me refute the idea that AI coding is just another rung up on the programming abstraction ladder. Depending on how much you delegate to AI, I don't think it's really programming at all. It's project management. That's not a bad thing! But it's not really still programming.

Even just in the context of my human team, I feel less mentally engaged with the code. I don't know what everything does. (In principle, I could know, but I don't.) I see some code written in a way that differs from how I would have done it. But I'm not the one working day-in, day-out with the code. I'll ask questions, make suggestions, but I'm not going to force something unless I think it's really super important.

That said, I don't 100% like this. I enjoy programming. I enjoy computer science. I especially enjoy things more down the paths of algorithm design, Lisp, and the intersection of programming with mathematics. On my team, I do still do some programming. I could delegate it entirely, but I indulge myself and do a little bit.

I personally think that's a good path with AI too. I think we're at the point where, for many software application tasks, the programming could be entirely hands-off. Let AI do it all. But if I wish to, why not indulge in doing some myself also? Yeah, I know, I know, I'll get "left behind in the dust" and all of that. I'm not sure that I'm in that much of a hurry to churn out 50,000 lines of code a day; I'm cool with 45,100.