Comment

Comment by robbomacrae

SCB is definitely an underrated benchmark. For me the unique selling point is that it more closely mirrors software development by not stopping after a single task. The agent has to keep code clean. The only disadvantage is all the problems are greenfield and not git inited so the agents don’t make use of git diffs.

I’ve used SCB as part of my assessment of agent skills (superpowers, GSD etc) https://orcabot.com/labs/do-skills-improve-coding-agent-accu...

There is a small but growing community on discord for discussing SCB so if interested please join https://discord.gg/BrC4BA9sVj

parent robbomacrae 3 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton