akshaysg

If you see my reply, I would absolutely love it if you could email [email protected] with some details about your desired use cases!

Thank you for the kind words!

RE Pricing: Because of the high volume of our customers, this is quite necessary unfortunately. We can definitely shift down costs as existing intelligence becomes cheaper though.

I think that AI coding is only going to grow in usage. I also think that AI more-or-less writes working code that is faithful to the user's prompt.

At the same time, AI does not write code that's easy for humans to review, as it writes very verbosely and with the goal of getting the job done vs producing readable code.

I really don't think humans should be reviewing the absolute mountains of LLM-produced code; doing so would only exhaust someone's cognitive budget.

Therefore, I think that bug finding should be left to AI to review, while architectural decisions should be reviewed by humans.

Yeah I agree it's a shame. Unfortunately coding has changed fast and I was not confident that the editor was in the correct direction with AI coding becoming so prevalent.

I think there is a lot of value with "reconnecting" with your codebase, so I do have some plans to bring the core concept of Haystack back in one form or another.

I think Haystack offers:

1. A centralized review mechanism for a team or org that operates on coding agent conversations in addition to diffs (and the codebase). It evaluates multiple different variables (e.g. how sensitive are the changes, how much did the author do to derisk, and what did the author's coding agent gloss over) and helps enforce your team's guidelines moreso than just an individual's prompt

2. Adversarial review that operates in addition to other AI review agents (e.g. BugBot, or Greptile) and filters any comments to only the things the author cares about. This helps cut down on the "AI reviewer battleground" that is present in pull requests

3. A review interface that allows human reviewers to quickly understand what the author did to verify their changes and focus on the author's design decisions

We actually jury-rigged all of this together before building Haystack, but found that it doesn't scale to the team level (since every individual has their own ideas/opinions of what constitutes a human review).

We also found that reviewing through purely Claude Code/Codex was slow and difficult because stuff like author traces are not pre-processed and you have to get your agent to specifically explore/understand them.

I've been thinking a lot about this!

Redoing the work as smaller PRs might help with readability, but then you get the opposite problem: it becomes hard to hold all the PRs in your head at once and keep track of the overall purpose of the change (at least for me).

IMO the real solution is figuring out which subset of changes actually needs human review and focusing attention there. And even then, not necessarily through diffs. For larger agent-generated changes, more useful review artifacts may be things like design decisions or risky areas that were changed.

Will add this then!

We use a mixture of LLMs (e.g. to identify the important variables to trace), language servers (to understand where a variable is originally defined and where it's used).

We may go back to code visualization for editors, but after testing it with a lot of engineers, we found that it was too confusing for code reviews. The main issue was just how alien the interface is!