Comment

Comment by aethelyon

I gave GPT-5.6 Sol, Fable 5, Grok 4.5, Sonnet 5, and GPT-5.5 the same greenfield spec: build the Basecamp 5 frontend and API. Fable won both tracks at $85.87 in 2:06:40. Grok reached 84% of Fable's frontend score and 87% of its backend score for $9.30 in 36:48. Five reruns exposed meaningful variance: the best run beat the median by up to 0.46 points, while Sol’s best frontend beat its first score by 0.72. The full report shows where each model excels, where it is okay, and where it fails.
parent aethelyon 3 months ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton