> LLMs are always hallucinating that they've found the most severe bug in software history.
That is a function of the training data/method, like many of AI's problems. This doesn't stop it being a problem of course, but it shouldn't be a surprise.
Big bugs, like heartbleed, over the years have had breathy websites going over how serious the problem is because the problem was that serious, so the models associate that sort of wording with bug reports. Less serious bugs don't get the same excited treatment, repeated in many many places, so more measured descriptions figure in the training data relatively less.
That is a function of the training data/method, like many of AI's problems. This doesn't stop it being a problem of course, but it shouldn't be a surprise.
Big bugs, like heartbleed, over the years have had breathy websites going over how serious the problem is because the problem was that serious, so the models associate that sort of wording with bug reports. Less serious bugs don't get the same excited treatment, repeated in many many places, so more measured descriptions figure in the training data relatively less.