robbie-c
Joined 11 years ago
Joined 11 years ago
That requires having enough good new ideas for everyone to work on, and sadly a lot of companies don't have those
The best way I have found to think about this is that consciousness is a property of a collection, like temperature. One atom does not have a temperature but if you have enough of them together then it's a useful enough property to talk about. Similarly, one human neuron does not have consciousness but if you put enough of them together then they do.
I'm begging people to take the time to write things themselves rather than getting Claude to write for them.
If you want human effort from readers, please put in human effort while writing.
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.
I want an expert opinion, if I just wanted to ask an LLM I have my own.
How can this article not mention the KV cache even once?
There’s no need for that. I’ve done plenty.
I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it?
Edit: this specific title has been deleted from the article. That was not my point! Please put in more effort into writing things that you want others to read! Rather than putting in low effort but being better at hiding it.
If you don't use a subscription, and pay per token instead, you can easily move to another harness.
In the future, we'd make changes by modifying the ANTLR parser first, then using the same approach as in the blog post to get the new parser to parity. We have no plans to get rid of the C++ parser as an oracle!
I think startups have an easier time for 2 reasons: They are innovating on the product side already, and probably were already capacity constrained for work on the top ideas. They also have a high proportion of tech debt which never bubbled to the top of effort vs reward (competing against product work) before but would still be valuable to fix. E.g. https://posthog.com/blog/sql-parser was on the HN frontpage a few months ago, and this work would never have happened in the before times