akane8
Joined 10 years ago
Karma
14
Submissions
18
Member since
May 14
Recent submissions
Comment · 5 months ago
Empirical evidence that TDD actively hurts agent performance measured by ProgramBench, while drastically increasing cost
Comment · 6 months ago
OPINIONS.md is a compact, living map of what you actually believe. I manage mine with the Hermes agent (OpenClaw would work too) and GPT 5.5. It is a markdown file in my dotfiles, updated daily by a cron job. It reads everything I write on the Internet, extracts durable opinions, merges them into the existing structure, and commits the result back to GitHub. I did it as an experiment, but it turned out to be a lot more helpful than I expected. I can now interrogate my own beliefs, and have them available as context to all my agents.
Comment · 6 months ago
haven't thought of other use yet! would be curious to hear if anyone sees another use case
Comment · 6 months ago
when the agentic review finds 7 problems and you only care about 4, and want to add some instructions for how the agent should fix them, this kind of interactive back-n-forth is very jarring to do in pre-commit hooks.
Comment · 6 months ago
great questions! the goal is to kill as much slop as possible before it becomes a PR and potentially waste other human reviewers’ time.
the feedback loop does exist and there’s a smooth review->fix loop there
Comment · 7 months ago
Yes it totally can. I started with a plain bash script myself :) But as complexity grew it just became harder and harder to make it work consistently under different conditions. Nix solved it beautifully.
Comment · 7 months ago
Technically MCP tools can be called in parallel as well, but it seems the agent is generally less likely to do so.
In the evaluation I did account for prompt caching, so the multi-turn penalty is already minimized, yet it was still significant enough to make the difference.
Comment · 7 months ago
MCP or CLI? There's a massive debate happening in the AI community around what's the best way to expose tools to agents. For curiosity, I spend the weekend doing an independent study and benchmarked Github CLI, MCP, MCP with Tool Search and MCP with Code Mode, all with real data and practical tasks.
Numbers are quite interesting. A few key findings -
1. Github MCP is 2–3x more expensive to use than CLI. Probably not surprising. There’s almost no practical reason to use their MCP except for some of the different handling of security
2. Tool Search saves upfront tokens but spends them on extra turns. Whether that trade-off pays depends on task complexity. ToolSearch also introduces a new failure mode due to imperfect search accuracy.
3. Code Mode is the cheapest way to use MCP, but still 2x more expensive than CLI, and it’s very slow. Code-mode also introduces a unique failure mode when the agent writes buggy code or poor error handling.
4. It's possible to push CLIs further towards higher success rate at lowest cost and latency with a principled design approach that treats agent ergonomics as first class concern. I detailed this in https://axi.md.
Benchmark harness, results and the reference implementation of gh-axi are all open sourced at https://github.com/kunchenguid/axi.