Comment Comment by totetsu “We ran 153 autonomous runs across 18 frontier models on the nanoGPT optimizer speedrun.”Uh.. okay.. but whats a run… read blog“We want to measure how well frontier models can conduct research….””we ran 153 autonomous runs on the nanoGPT optimizer speedrun across”Okay but what is a optimiser run and what connection does it have to being good at research?“For comparison, Anthropic's internal automated AI R&D evaluation optimizes a model on a CPU node,”So I should go look what Anthropic was doing to understand?Why not just explain what it means in their blog.. parent totetsu 2 months ago
Uh.. okay.. but whats a run… read blog
“We want to measure how well frontier models can conduct research….””we ran 153 autonomous runs on the nanoGPT optimizer speedrun across”
Okay but what is a optimiser run and what connection does it have to being good at research?
“For comparison, Anthropic's internal automated AI R&D evaluation optimizes a model on a CPU node,”
So I should go look what Anthropic was doing to understand?
Why not just explain what it means in their blog..