Azkar
Joined 13 years ago
Joined 13 years ago
I think this is a great question, and it probably needs to be re-evaluated every time models change. I added each of these to my workflow because I found that as my project grew in size and lines of code, single agents were missing large pieces of context and needed to be fed a lot of guidance. You can also determine how many/which type of agents to spawn based on the size/risk of the change. Ultimately it's probably a trade-off between speed/safety. For short term projects, a single agent is fine.
> I have a hard time understanding how that is being achieved. Are you finding yourself naturally gravitating toward that type of velocity and having to push back on it?
I can see it. I do have to push back on the urge to want to do more things in parallel. However, the more I use these agent workflows, the more I trust them, the more autonomy I give them. I have no problem passing a well written ticket to an agent and trusting it to build/test/merge/validate the change in prod.
It goes on to describe what counts as a claim, how to verify the claims, and how to respond. It responds to each claim with verified, unverified, or contradicted.
The skeptic agent has been the most high value thing I've added to the workflow.
Every feature I build uses a skill that does the following:
1. Read a ticket and get context on the task. The ticket was probably written by another agent after a conversation with myself about what is happening/needs to happen, etc.
2. Plan the task, asking for clarification where needed
3. Pressure test the plan, and validate the plans logic (subagents)
4. Implement
5. Runtime/local validation
6. Post PR, review it using applicable agents (database, security, code, prose...)
7. Fix PR based on feedback
I generally get excellent results out of this process, and I cannot imagine trying to orchestrate this without a skill. But I also can imagine my workflow isn't tuned to be super usable for anyone else.
I think we're starting to see that across the entire industry. Leadership is easy when times are good. The job has gotten very hard.
1. Solid code reviews. Anyone of our developers can halt a code review for any reason. We require 3 approvers on each review. Sensitive areas require reviews from people familiar in that area. We also have tooling that allows us to generate amounts of test data in dev that is similar to prod loads. This helps us catch a lot of time bombs.
2. Feature toggles to decouple deploy of code from release of code. This allows us to test our code in production before turning it on for customers. It also allows us to slowly rollout a feature and watch how the code behaves. This also gives us a kill switch to turn off the code if it is bad.
3. An incredibly robust testing pipeline. It takes about 50 minutes from commit to production deployment. We can also deploy previous containers very quickly for situations that require it.
This doesn't solve all of our problems. Some changes cannot go behind feature toggles (DB migrations, dependency upgrades, etc). But we do pay a lot of attention to design and rollout plans for database migration changes and such.
All of these things come at an extra cost to us, but it allows us to move quickly when we need to. But we're in a lot better place than we were when we were trying to do weekly releases. We have a good mix of team experience (sr vs jr) - and have a lot of discipline in our software engineering practices. We still have problems like I said, but these strategies have greatly improved our ability to deliver software.
That's been my experience as well. When a manager says they try to "shield you from the bullshit" it's just a lack of transparency that leads me to making my own (often worse) assumptions.
It seems like none of this actually matters until we can solve that problem.
Looking at my recent usage, it’s the second most after the feature orchestration itself. It will also depend on how many claims it has to verify and what I’m building. It’s the most valuable piece of glue that I’ve found when passing things between other focused agents.
My experience is with Claude, but I expect other frontier models would work just as well.