Comment

Comment by crabbone

Not going to work.

1. The reason OOP complained about AI code being shipped without comprehension is because they want to ship of it more and faster. If they had time, they'd review the code. Writing tests requires a lot more time than what people usually spend on code reviews (unfortunately, but that's how it is). And if you think you can generate tests, again, by using AI, then, the bad news are that now you have both the code you don't understand and the tests you don't understand. Don't forget that running tests also takes time, in some cases it takes longer than to write code. So, again, your benefits obtained from AI-generating code won't materialize.

2. Even if magically you were able to procure decent-quality tests (without understanding what you are testing!) There are still problems not addressed by testing s.a. reusability. If you don't understand what the code does, how can you possibly reuse it for something else? Perhaps a small modification or a generalization of the existing code would allow you to reuse the mostly tested codebase for the new feature, but since you don't know what it does, you've missed that opportunity, and, again, the AI-generated code came second in the race to ship the product faster (also, you've got a worse product).

* * *

The question of "how to deal with code we don't understand?" is by no means a new one. Dijkstra's structured programming was a response to precisely this question. His argument showed that programmers can't even anticipate all the potential problems to be found in a 16-bit adder and that exhaustive testing is prohibitively expensive. Therefore, he argued, we should restrict the kinds or programs we write to those that are easier to understand (and that's what he called "structured" programming).

Perhaps, we can find a similar sort of harness for AI writing code, where we can impose boundaries it cannot cross that ensure understanding. I have no idea what that can look like, but I'm absolutely sure that testing isn't it.

parent crabbone 11 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton