thomashabets2

Well, it was a mix. Win32, then the Lotus family just refused to conform.

And some others. IP? No, we are a NetWare IPX network!!

Author here.

Of course I would also prefer a warning, or at least a guarding trap instruction. And maybe compilers should try harder to be friendly in this way.

But they can't be perfect, because the program specified something syntactically valid, but semantically invalid. The module/compiler pass that could theoretically detect it may simply not be issued the necessary information. E.g. a solver may be given the input that a value is in a given range, and from that simplify an operation. If the input is wrong, so is the output.

Reminds me of:

    On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

In the example from the article, though, there could be a pass that simply eliminated dead code. That pass is not a parser of the raw code, so how is it to know that actually, the "dead code" isn't?

And a standard that would require dead code elimination to replace not with empty, but with a trap instruction, could go very wrong. You don't want `if (false) { … }` to become `if (false) { trap(); }`. You want it just removed, if a previous pass deemed it dead. Or do you want `int main() { for(;;) {cli_loop();} }` to force adding a trap in the unreachable code after the loop? That's a lot of generated trap instructions.

So in general this means solving the halting problem. Yes, specifically in the post's example, "surely a sufficiently smart compiler could detect something this obvious". Maybe so, but how often is the bug actually this obvious?

Free Anti Matter!

Anti Matter did nothing wrong!

To elaborate on the other reply to your comment: In Rust, if you don't use `unsafe` and still manage to trigger UB, then that's a bug in the compiler or the library code you called (which in turn has to have used unsafe somewhere to trigger UB).

So the fact that `unreachable!()` does not require the use of the keyword `unsafe` means that it can't be like C++ `std::unreachable()`.

Author here.

Yes, into the shame cube I go. It doesn't change anything, and it applies equally to C and C++, but you are correct. Thanks.

C++26, IIRC, adds "no time travel" to C++. But note that your example still creates exploding chocolate, so what do you mean by "immune"? C compiler with -std=c23 -O1 produces literally the exact same assembly output.

> This is sort of true, except that behavior can be undefined in the standard but defined in your implementation.

But then it's not UB, and not the topic at hand. So I wouldn't call it "sort of true" for a statement about UB, because it doesn't apply to not-UB.

It's also different in a consteval context. But that doesn't make it less true outside consteval context.

Author here: yes and no.

UB can also be things that the compiler is unable to express. It's NOT "haha, now I can do anything", it's like asking what the correct answer is to a logically inconsistent question. The compiler cannot even express the question.

Let's say the compiler encodes some GC information in the low two bits of int pointers. What does it mean if a function is called with an unaligned pointer, then? Mu, unask the question. It's UB, and no diagnostic required nor even feasible.

Vibe coding may produce a lot of code, but I'd say LLMs can already find more than 99% of bugs, and I extrapolate that they therefore won't add them to living systems.

But even if new code gets bugs, we run a bug finding pass on the delta. So the vulnporalypse will end.

And in my prompts I've seen LLMs find complex interaction bugs that a human would basically never find (and the bug was added by a human).

You say polluted by vibe coding. But in terms of bugs, especially in the last like 6 months, you could argue that humans are the polluters.

E.g. while I've seen "ai deleted our database", but if what you're saying is true, we should be seeing "AI added this buffer overflow". But we don't, because humans are the polluters of that.

Yes, there have been "ai coded up client side password checking", but I've not seen that recently. Only like 6 generations and three years ago.

At some point it's like telling the cab driver to just follow Google maps directions. Like... sorry, cabbie (Black Cabs in London with their "the Knowledge" is what I'm thinking), your memorization, while impressive, is worse than the app.

The painter, while impressively skilled, is a worse documentarian than the camera.

And that point is probably behind us. At least for typing. And it's climbing up in abstractions.

It could lead to a mess, sure. But my favourite prompt on an LLM-untouched codebase is along the lines of "find UB and other bugs. If the fix is less than 5 lines, fix it and commit it. One change per commit". Then I of course check the commit history before pushing.

My experience over the last couple of years with this type of prompt is that eventually you run out of bugs to find (meaning you reach "I see what you mean, but I disagree").

And those fixes don't cause messes.

And a new model may find something new, but not even remotely the volume seen the first time. This indicates to me that the supply of bugs is finite, and after this period of house cleaning, it should go down.

But anyway, a mess (decoherence) doesn't necessarily mean an exploitable mess.

Reading that good article, I find myself thinking that we did not need LLMs to get decoherence in complex software or systems. At least LLMs don't have personal fiefdoms, egos, and company politics adding to encouraging decoherence.

> The bugs will only disappear until the model that was used to security test the generated code becomes obsoleted.

I think you're ignoring the distinct possibility that even if the code is not formally verified, LLMs could simply stop writing bugs, or this kind of bugs. No matter how smart an attacker AI, it cannot exploit bugs that are simply not there.

But hell, even formally verified ones can only verify back to the spec. There are plenty of examples of bugs in formally verified software.

I place a higher likelihood on LLMs winning (producing fewer bugs than formally verified) over the long run. Mostly because verified software can, IIUC, only verify back to the spec. Whereas LLMs can go "uh, your spec itself has a bug, you really want me to codify that?".

Luckily we have newer models going back and fixing these things.

In terms of the set of all bugs, the AIs from 2022-2023 pale in comparison with the amount of bugs written by hand in previous half century, even if 2022 AIs were worse than the average programmer.