Nizoss
Joined 9 years ago
Joined 9 years ago
Hooks do wonders here. The payload contains a lot of information about the pending action the agent wants to make. Combine that with the most recent n events from the agent’s session history and you have a rich enough context to pass to another agent to validate the action through the SDK.
This way the validation uses the same subscription you’re logged in to, whether you’re using Claude Code, Codex, or Copilot. The validation agent responds with a json format that you can easily parse and return, allowing you to let the action through or block it with direction and guidance. I’m genuinely impressed by how well this works considering how simple it is.
You can find my approach here: https://github.com/nizos/probity
I created it to address this exact issue. It is a vendor-neutral ESLint-style policy engine and currently supports Claude Code, Codex, and Copilot.
It uses the agents hooks payloads and session history to enforce the policies. Allowing it to be setup to block commits if a file has been modified since the checks were last run, disallow content or commands using string or regex matching, and enforce TDD without the need of any extra reporter setup and it works with any language.
Feedback welcome: https://github.com/nizos/probity
TDD Guard was built when Claude Code was the only one to offer hooks. Plugins didn't exist and the models were weaker, so the validation context and instructions took more work to get right. This is why it ended up requiring test reporters for different languages.
I have started a new project that does the same TDD enforcement, also through hooks, but without reporters. It works with any test runner, and it is vendor-agnostic, it works with Claude Code, Codex, and GitHub Copilot. The validator also sees recent session history which helps it handle cases like refactoring better.
The TDD instructions are still pretty basic compared to TDD Guard's, which have been dogfooded for a year. One thing I noticed while testing across agents is that some follow TDD a lot better than others, Codex struggled the most with the basic instructions.
Feedback welcome:
This setup works great especially when you work with multiple agents or sessions in parallel and don’t want to be babysitting TDD. You just know that no TDD shortcuts or violations will be made and can focus on the solution instead. Agents are good at internally justifying shortcuts and lowering what’s good enough as the session goes. You can notice this when you ask them to review their own work compared to when asking a new session to review the changes. The difference is stark.
What’s interesting about the TDD instructions I dogfooded for this is that there is a lot that is implicit about how to interpret operations in terms of TDD violations. For example, earlier versions of the instructions had the validation agent block multi-step refactor changes because there was no guarantee to them that further changes will follow. It would also block changes when a definition is removed while it is still being called. The reasoning is that the code will no longer build and thereby not fulfill the ”refactoring is allowed under green”. Improving the wording and clarifying the process helped from this unwanted false blocks.
If you want to give this approach a try, you’ll find it here. I’m the author and I’m happy to and any further questions: https://github.com/nizos/probity