This rings similar to a recent post that was on the front page about red team vs. blue team.
Before running LLM-generated code through yet more LLMs, you can run it through traditional static analysis (linters, SAST, auto-formatters). They aren’t flashy but they produce the same results 100% of the time.
Consistency is critical if you want to pass/fail a build on the results. Nobody wants a flaky code reviewer robot, just like flaky tests are the worst.
I imagine code review will evolve into a three tier pyramid:
1. Static analysis (instant, consistent) — e.g using Qlty CLI (https://github.com/qltysh/qlty) as a Claude Code or Git hook
2. LLMs — Has the advantage of being able to catch semantic issues
3. Human
We make sure commits pass each level in succession before moving on to the next.
Before running LLM-generated code through yet more LLMs, you can run it through traditional static analysis (linters, SAST, auto-formatters). They aren’t flashy but they produce the same results 100% of the time.
Consistency is critical if you want to pass/fail a build on the results. Nobody wants a flaky code reviewer robot, just like flaky tests are the worst.
I imagine code review will evolve into a three tier pyramid:
1. Static analysis (instant, consistent) — e.g using Qlty CLI (https://github.com/qltysh/qlty) as a Claude Code or Git hook
2. LLMs — Has the advantage of being able to catch semantic issues
3. Human
We make sure commits pass each level in succession before moving on to the next.