fitz2882

Joined 9 months ago

Karma
10
Submissions
11
Member since
Feb 3
I'm Dave, an AI dev in Toronto.

Building LoopGain, an open-source tool that measures when an agentic loop has stopped improving (loop gain / Aβ, borrowed from control theory) and stops it there instead of burning credits to max_iterations. Stops at convergence, rolls back on divergence.

loopgain.ai

Happy to talk agents, control theory, or anything in between.

View on Hacker News

Recent submissions

Comment · 3 months ago
Hey, sorry I didn't see your comment until now. Yes I just added a Claude Code plugin that allows LoopGain to work with /goal.

/goal's checker knows when it's done but not when it has stopped improving. The plugin sets a Stop hook that, after each turn, runs your own verifier (such as failing-test count) and stops the loop when LoopGain sees it's converged or stalled instead of a guessed cap.

- two install commands:

   - /plugin marketplace add loopgain-ai/loopgain-plugin

   - /plugin install loopgain
Any skill that iterates can just call the LoopGain library from a small script to decide when to stop instead of a hardcoded cap. I do this in my own skills. It recurses until the number of unresolved pieces stops dropping, no fixed depth limit.

You can also point any AI agent to the repo so it learns how LoopGain works and it will be able to help you set it up how you like. It's open-source.

Comment · 3 months ago
Hey everyone, I just want to clear some things up here based on the comments:

I (Dave) am a real person, and yes I used AI to help me with every part of this project, including my initial comment introducing it here. That was a mistake, I should have just written it myself. I am a self-taught developer who is passionate about AI development and curious about seeing what I can do with it. I come up with my ideas and test them with the help of AI. Then I have AI help me understand it. This all started because I was building multi-agent systems, just experimenting, and I noticed that they looked to me like electrical circuit diagrams. So it made me curious if there was any cross over between the laws of electrical engineering and how it might apply to agentic systems. With AI today you can pursue any idea you have, so I have fun by seeing what happens when you take concepts from one established field and apply them to something else. I had AI do deep research on electrical engineering concepts, and asked it to research how those might be applied to agentic systems. It came up with interesting ideas, so I had it plan scientific experiments on ones that looked promising, and the one that looked like the most obvious application is what turned into LoopGain. With the help of AI I developed it into a working system, created a website and company around it, made the project open source, wrote blog articles about the experiments, then asked AI how to best share it with the world. I love developing useful tools and researching the edge of AI capabilities, so that is what I am doing here. Thanks for reading.

Comment · 3 months ago
Hey gavmor, thanks for your comment! Let me know if you need any assistance, I would be happy to help or get any feedback at [email protected]
Comment · 3 months ago
Hey, please see my response to lochnessduck, thanks
Comment · 3 months ago
Hey, thanks for your comment. This is an idea that I thought up, researched, developed, and in the end built a useful, working, open source tool. Yes I used AI to assist me in the process, it's a complicated subject. But I take your point that it would be better to write in my own words. You're right, lesson learned.

Anyhow, whether I used a tool like AI to assist me or not, I am here to share something I built that I hope others will find useful. I am building what I am passionate about.

Comment · 3 months ago
Hey HN — Dave here. While building a multi-agent system a couple of years ago, I kept noticing that the diagrams of agents passing feedback to each other looked just like the electrical circuit diagrams from control theory. I got curious whether the established math transferred, and to my surprise it did. LoopGain is the first product of that research: an open-source library that replaces the max_iterations=N cap on agent loops with an actual measurement of whether the loop is still improving.

The cap is how nearly every verify-revise loop gets stopped today, and it's wrong in both directions. Stop too early and you clip a loop that was still improving. Stop too late and you pay for iterations after the loop already found its best answer — and ship the final attempt, which is sometimes worse than one it already had.

On a 2,000-trial benchmark (paired real-API runs, five loop patterns across six framework adapters, three model providers, pre-registered protocol with kill criteria), LoopGain cut total API spend by 92.8% vs max_iterations=20 ($27.05 → $1.94) and median wall-clock ~15x (30.9s → 2.1s). A cross-vendor judge preferred LoopGain's outputs on the weighted average (0.678 across 1,800 pairwise comparisons) — mostly because LoopGain returns the iteration with the lowest error it saw, while a fixed cap ships whatever the final iteration produced. The raw data and methodology are public, the full run is browsable at dashboard.loopgain.ai/benchmark, and there's a replay of a real benchmark run (all measured, no simulation) at dashboard.loopgain.ai/demo.

How it works: each iteration, LoopGain takes the ratio of the current error to the previous error — the loop's empirical loop gain (Aβ, borrowed from control theory). Aβ<1 means the error shrank — the loop is improving. Aβ≥1 means it held or grew — the loop is stuck or making things worse. A trajectory classifier reads the recent Aβ values, labels the loop (FAST_CONVERGE / CONVERGING / STALLING / OSCILLATING / DIVERGING), and decides whether to keep going, stop here, or stop and roll back to the lowest-error output so far.

Integration is a few lines around any loop that produces an error signal:

    from loopgain import LoopGain

    lg = LoopGain(target_error=0.0)
    output = generate(task)                # first attempt

    while lg.should_continue():
        errors = verify(output)            # e.g. count of failing tests
        lg.observe(errors, output=output)  # the only LoopGain call in the loop
        output = revise(output, errors)

    result = lg.result   # .best_output, .outcome, .convergence_profile, .savings_vs_fixed_cap
Honest limits, because they matter more than the headline: LoopGain detects convergence, not correctness — it inherits your verifier's blind spots. I re-graded my own benchmark and 4.5% of "converged" code-gen runs passed every check the loop ran but failed a fuller held-out test suite. And savings depend on workload: failure-heavy loops save ~78–84%, not 92.8%. There's a writeup on designing verifiers strong enough to trust on the blog.

Apache-2.0, pip install loopgain. Adapters for LangGraph, CrewAI, AutoGen, LangChain, OpenAI Agents SDK, and the Claude Agent SDK; the raw API works for anything with a measurable error.

What I'd really love from HN: if you run production agent loops, I'm interested in whether the stop decisions match what you see empirically — and what your loops' error signals actually look like, since the verifier is what makes or breaks the stop. Happy to answer anything.

Comment · 4 months ago
Hey HN — Dave here. While building a multi-agent system recently, I kept noticing that the diagrams of agents passing feedback to each other looked just like the electrical circuit diagrams from control theory. I got curious whether the established math transferred, and to my surprise it did. LoopGain is the first product of that research: an open-source library that replaces the max_iterations=N cap on agent loops with an actual measurement of whether the loop is still improving.

The cap is how nearly every verify-revise loop gets stopped today, and it's wrong in both directions. Stop too early and you clip a loop that was still improving. Stop too late and you pay for iterations after the loop already found its best answer — and ship the final attempt, which is sometimes worse than one it already had.

On a 2,000-trial benchmark (paired real-API runs, five loop patterns across six framework adapters, three model providers, pre-registered protocol with kill criteria), LoopGain cut total API spend by 92.8% vs max_iterations=20 ($27.05 → $1.94) and median wall-clock ~15× (30.9s → 2.1s). A cross-vendor judge preferred LoopGain's outputs on the weighted average (0.678 across 1,800 pairwise comparisons) — mostly because LoopGain returns the iteration with the lowest error it saw, while a fixed cap ships whatever the final iteration produced. The raw data and methodology are public, and the full run is browsable at dashboard.loopgain.ai/benchmark.

How it works: each iteration, LoopGain takes the ratio of the current error to the previous error — the loop's empirical loop gain (Aβ, borrowed from control theory). Aβ<1 means the error shrank — the loop is improving. Aβ≥1 means it held or grew — the loop is stuck or making things worse. A trajectory classifier reads the recent Aβ values, labels the loop (FAST_CONVERGE / CONVERGING / STALLING / OSCILLATING / DIVERGING), and decides whether to keep going, stop here, or stop and roll back to the lowest-error output so far.

Integration is a few lines around any loop that produces an error signal:

  from loopgain import LoopGain
  lg = LoopGain(target_error=0.0)
  output = generate(task)                # first attempt
  while lg.should_continue():
      errors = verify(output)            # e.g. count of failing tests
      lg.observe(errors, output=output)  # the only LoopGain call in the loop
      output = revise(output, errors)
  result = lg.result   # best_output, outcome, convergence_profile, savings_vs_fixed_cap
Honest limits, because they matter more than the headline: LoopGain detects convergence, not correctness — it inherits your verifier's blind spots. I re-graded my own benchmark and 4.5% of "converged" code-gen runs passed every check the loop ran but failed a fuller held-out test suite. And savings depend on workload: failure-heavy loops save ~78–84%, not 92.8%. There's a writeup on designing verifiers strong enough to trust on the blog.

Apache-2.0, pip install loopgain. Adapters for LangGraph, CrewAI, AutoGen, LangChain, OpenAI Agents SDK, and the Claude Agent SDK; the raw API works for anything with a measurable error.

What I'd really love from HN: if you run production agent loops, I'm interested in whether the stop decisions match what you see empirically — and what your loops' error signals actually look like, since the verifier is what makes or breaks the stop. Happy to answer anything.

Comment · 9 months ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton