> 25% of developers estimate that 1 in 5 AI-generated suggestions contain factual errors or misleading code.
Seem incompatible with "often full of noise", to you?
I can't speak for factual errors, but I'd say less than 20% of the code ChatGPT* gives me contains clear errors — more like 10%. Perhaps that just means I can't spot all the subtle bugs.
But even in the best case, there's a lot of "noise" in the answers they give me: Excess comments that don't add anything, a whole class file when I wanted just a function, that kind of thing.
* Other LLMs are different, and I've had one (I think it was Phi-2) start bad then switch both task *and language* mid-way through.
> 25% of developers estimate that 1 in 5 AI-generated suggestions contain factual errors or misleading code.
Seem incompatible with "often full of noise", to you?
I can't speak for factual errors, but I'd say less than 20% of the code ChatGPT* gives me contains clear errors — more like 10%. Perhaps that just means I can't spot all the subtle bugs.
But even in the best case, there's a lot of "noise" in the answers they give me: Excess comments that don't add anything, a whole class file when I wanted just a function, that kind of thing.
* Other LLMs are different, and I've had one (I think it was Phi-2) start bad then switch both task *and language* mid-way through.