breckenedge
Joined 14 years ago
Joined 14 years ago
But I guess you mean the discovery part? Yea, agents are not going to totally replace that anytime soon. Though, Claude voice mode is pretty good these days, so I’m sure it’s on their mind.
Not sure what you mean by “stay the course.” I think the winners will be those with the most effective marketing strategies. Surely that will require automation.
You’re not running tests in CI? The agent should only run the minimum tests necessary and then idle and wait for feedback from a CI run.
Yesterday, I ran an identical bug identification dataset from two weeks ago, saw a 50% drop from a few weeks ago, putting Sol on the same level as Luna. Sol had been finding 40-50 bugs per set, then dropped to 25, matching Luna’s performance. Not enough to establish a pattern, but enough to raise eyebrows.
Our review workflow is public if you want to peruse it, dataset isn’t. The process isn’t really stabilized yet either as I have to balance running this against limited budgets.
https://github.com/BiggerPockets/.github/blob/main/.github/w...