While I, like almost everyone, doesn't love reading LLM output, code review is an excellent use for LLMs. You know what I like less than reading LLM output?
- Slow PR review
- Nitpicks on style, not substance
- Async waiting for review or re-review after making changes, harder to stay in what passes for a "flow state" nowadays
- A lot of a ceremony around straightforward changes
We use LLM reviews at my company and it's caught things human reviews have not (as in in the early days we human-reviewed code and then did tests running the LLM afterwards and it caught things we didn't). It can review code at any time of the day and re-review it just as quickly as you can address the feedback.
I predict we (my company) are less than a year out from having a fully automated pipeline for reviews with only humans getting called in when it touches certain parts of the code. We already have a crude system for detecting certain file paths that we want a human to review if touched and we tweak that list over time as needed.
- Slow PR review
- Nitpicks on style, not substance
- Async waiting for review or re-review after making changes, harder to stay in what passes for a "flow state" nowadays
- A lot of a ceremony around straightforward changes
We use LLM reviews at my company and it's caught things human reviews have not (as in in the early days we human-reviewed code and then did tests running the LLM afterwards and it caught things we didn't). It can review code at any time of the day and re-review it just as quickly as you can address the feedback.
I predict we (my company) are less than a year out from having a fully automated pipeline for reviews with only humans getting called in when it touches certain parts of the code. We already have a crude system for detecting certain file paths that we want a human to review if touched and we tweak that list over time as needed.