ohans
Joined 6 years ago
Joined 6 years ago
So, before reading the code diff on PRs, we can judge from visually rich architecture and data flow diagrams. This helps us keep a clear mental model of how every part of the system is evolving.
We built a new renderer to support some of the rich visualisations, and curious if it's helpful for others
The consensus, in order of how much they write about it mostly: architecture and repo layout, how to test, build commands, dos-and-don'ts, PR etiquette, and code style.
The surprise was tone, and the lack of non-technical constraints.
90% write in must/always/never, and there are 784 explicit "don't" bullets, most of them oddly specific.
It's almost like you can tell exactly which mistake an agent made in each repo.
Some are hilarious: "Do not claim that an interrupted or timed-out test passed" takes the gold for me.
The shortest is 35 words, one rule from Neovim.
> AI Disclosure: If AI was used in any way for a commit, add an AI-assisted: <tool name> trailer to the commit message. If the user commits manually, remind them to add it.
The most popular headings by far were: testing, commands, project overview, and architecture
There's a lot more interesting deets like the average length, nextjs' "Do NOT add "Generated with Claude Code" ..."
Is your AGENTS.md very different to these?
That being said, the reason I think cloud AI review companies (that review on PRs) work is that at a large org you need a way to ensure that the review indeed happens. And you can't blindly trust that the dev did this locally.
We're a small team, so shifting left works. Just review locally. If you're a large org, that's a different problem space
Since it allows us to automate regression testing with visual QA agents that test the app like a real user (mostly through Coldtea, which we build)
P.S: Been meaning to have calls recorded/transcribe. How do your agents retrieve the vast info in these transcripts? By that I mean is it effective?
AI agents have dominated Twitter/X the past few days: Claude Code, Remotion, Cursor, etc. But most developers are running these locally. Terminal agents on your machine.
That's useful, but it's a different game when you're building AI apps for users. The moment your agent needs to actually execute something e.g., run a Python script, process a PDF, generate files, you're suddenly building sandboxing infrastructure, managing VMs, handling file storage.
It's a massive distraction from your actual product. We built Bluebag to solve this for Vercel AI SDK users. Two lines of code, and your agent gets access to Skills that execute in managed sandboxes.
No Docker orchestration. No K8s. Dependencies, file handling, signed download URLs, all handled.
The post walks through some of the architecture (progressive skill loading, auto-provisioned VMs, multi-tenant isolation) and shows concrete integration examples.
Happy to answer questions about the approach or trade-offs we made.
Cheers
As for PR reviews, assuming you've got linting and static analysis out the way, you'd need to enter a sufficiently reasonable prompt to truly catch problems or surface reviews that match your standard and not generic AI comments.
My company uses some automatic AI PR review bots, and they annoy me more than they help. Lots of useless comments
I do think "logs are broken" is a bit overstated. The real problem is unstructured events + weak conventions + poor correlation.
Brilliant write up regardless
As always, curious to hear your thoughts