sergeyk

If anyone wants this kind of benchmark for their own codebase, happy to set you up with superconductor.com/benchmark

1. Import your own PRs 2. We find the original spec or infer the spec 3. Agents you select (eg claude code opus 5, codex gpt 6 astra high, pi kimi k3, etc) implement the spec (starting from the parent commit of the PR, with the git history is pruned so they can’t look up the implementation) 4. Three judge LLMs grade agent solution given spec, PR implementation, and a rubric 5. You see the scores

Check out superconductor.dev (I’m building it), if you want live app previews, docker-in-docker functionality, multiple agents in one mobile app, and more.

Our foundational design value was compute instance startup speed. We've made some design decisions and evaluated several "neocloud" providers with this goal in mind.

Currently, from launching an agent to that agent being able to run tests in our Rails docker-compose environment (and to the live app preview running), is about 30 seconds. If that agent finishes their work and goes to sleep, and then hours later you come back to send a message, it'll wake up in about the same time.

(And, of course, you can launch many agents at once -- they're all going to be ready at roughly the same time.)

This is exactly why we built superconductor.dev, which has live app preview for each agent. We support Claude Code as well as Gemini, Codex, Amp. If you want to check it out just mention HN in your signup form and I’ll prioritize you :)