Some people say they find LLMs very helpful for coding, some people say they are incredibly bad.
I often see people wondering if the some coding task is performed well or not because of availability of code examples in the training data. It's way worse than that. It's overfitting to diffs it was trained on.
"In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers."
Some people say they find LLMs very helpful for coding, some people say they are incredibly bad.
I often see people wondering if the some coding task is performed well or not because of availability of code examples in the training data. It's way worse than that. It's overfitting to diffs it was trained on.
"In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers."
https://arxiv.org/abs/2206.08896