jakelevirne

Joined 1 year ago

Karma

Submissions

Member since

Jan 18

Recent submissions

This is great to see. I've now fully embraced long-running agentic workflows with separate plan->generate->evaluate steps, all coordinated by an orchestrator. I've done this using Claude Code alone, which is very easy but costly using Fable alone. I've done this in a team-visible way using Linear and Cyrus https://specstory.com/tutorials/team-based-loop-engineering.

And lately, for cost savings I've been doing this via Claude Code orchestrated workflows that fan out to lower cost Pi.dev Kimi agents https://github.com/jakelevirne/pi-relay.

I think there's a lot to be said for having orchestrated goal-oriented workflows (loops/harnesses) that have their choice of agents. So it's nice to see that principle in play here. And strong goal/outcome definition is critical for success with these long running workflows, so helpful to see SpecFlow methodology baked in to the skill.

I think the biggest thing I've seen over and over as teams try to adopt this type of approach is weak testing/verification. Using agentic development it's very easy to have automated unit and integration testing. But what this approach really demands is acceptance testing and intent verification. Most people I know still do this part by hand, which means the loops can't be as long-running as they ideally would be. Have you thought about a deeper (more deterministic) verification approach, in addition to adversarial review from another agent?

(Note: I'm a SpecStory maintainer but didn't have anything to do with this Claramap Builder project)

Comment · 1 year ago

> Could you use it even on teams that aren’t using AI to generate code?

That's a great question that we haven't really talked about. The part of the AI codegen workflow that prompted this (no pun intended) is that it forces you to say/type aloud all your incremental intentions. It's the first time a tool has access to what was often previously inner monologue for most developers. But I know several developers who think aloud even without using AI (rubber ducking). But beyond just thinking aloud, using an AI code generator like Cursor Agent lets you express intent aloud and then easily change your mind (e.g. abandoning a code branch or using Cursor checkpoints)... the informal intent is right there next to the code for the first time.

Yes, comments could be that in theory. But in 70 years, I still don't think we've got a single fully commented codebase that includes fully documented product and technical intent.

Inferring intent is _really_ interesting. Though of course we have many examples where the code does not accurately express the intent (thank you, bugs). But could we bootstrap an intent store from existing code and then allow developers to validate and augment it with a more verbose process going forward? I think so.

Comment · 1 year ago

Exactly. We've now got help in the inner loop, authoring code. But the bottleneck just shifts. If you're using ai codegen to write production code, but then throwing it all over the wall at another engineer to review you're not getting the full productivity benefit. It's time to envision an ai first coding workflow that covers the whole development lifecycle.

Comment · 1 year ago

You're spot on. Sometimes with ai codegen we end up offloading some of the intentionality to the AI. They work surprisingly well at filling in the details for under-specified prompts. I think it's important that this tool capture not only the prompts and context given, but also the full set of responses back from the model. There are implicit requirements buried in those responses as well that are made explicit at the moment a developer decides to accept those changes.