I've submitted two PRs to fix bugs in open-source system tool last month. Both of these are targeted fixes to highly-specific bugs that existed in this software. The agents I used debugged the issue and implemented fixes and tests autonomously.
These problems looked slightly more difficult than bubble sort to me. Do you suppose these bugs existed in some other software and the agent only translated the fix between repositories? When the agent was debugging the software, were they simply following some tutorial they were trained on? Is this the 1% case? Or are the stochastic parrots regurgitating most probable tokens in approximately the right order to provide an illusion of work being done?
> In case you would say "but LLM-s learn on random source code from the internet"
These problems looked slightly more difficult than bubble sort to me. Do you suppose these bugs existed in some other software and the agent only translated the fix between repositories? When the agent was debugging the software, were they simply following some tutorial they were trained on? Is this the 1% case? Or are the stochastic parrots regurgitating most probable tokens in approximately the right order to provide an illusion of work being done?
> In case you would say "but LLM-s learn on random source code from the internet"
Of course they don't, it's not 2022 any more.