Joined 1 month ago
The main insight was that a multiple models arguing over a code change produces far better insights than a single model review.
While looking for practical applications, I realized coding is the space that needs this most because it has the highest AI agent adoption and the mistakes made by agents cost 30x more if they are discovered in production (added to the loss of customer trust). So, we developed Panel Review - it routes an AI coding agent's riskiest diffs, designs, and commits through a panel of four frontier models (e.g., OpenAI, Anthropic, Google, and xAI). Tool has review gates the agent cannot skip. We built these gates after learning the hard way that agents don't follow written instruction to do a review. In ten weeks of live production use, the panel found at least one critical or major issue in 46% of the reviews and 70% of the reviews resulted in the agent changing implementation based on panel recommendations. For example, one review caught a prompt injection hole in our own MCP server. Happy to answer any questions.
This sparked an idea for me to use Jev in my recently launched Panel Review tool that reviews the agent's riskiest decisions using a multiple models that argue and come to a consensus. see more - https://truverif.ai/panel-review. In order to determine risk, we use a text based classifier which I think could be replaced by Jev. Will still have to evaluate the privacy and cost considerations.
