Comment

Comment by dhorthy

Yeah and I started with subsets although I did not find them representative, eg I picked 6 challenges for a subset and fable scored 33%, while on the full challenge it landed closer to 14-15% range

The part II post linked in my top comment has a list of the challenges I used for that.

To run them, just point your model at the paper, it can find the repo, etc and with a key or sub, get it running

parent dhorthy 11 days ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton