Comment

Comment by scuppernong

https://frontierharness.org/ allegedly this is on the Pareto frontier, though I don't know how good of a benchmark this really is. it seems to focus on one-shot type tasks, whereas the real utility of one harness over another seems to make itself known in long running tasks.

also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.

parent scuppernong 29 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton