Comment

Comment by tintor

- OpenAI claims Astra beats all benchmarks (compared to Fable and Opus, except "Humanity's Last Exam (w/ tools)"): https://openai.com/index/gpt-6-astra/

- Artificial Analysis scores Astra (max effort) as 61 points on intelligence, behind Opus 5. https://artificialanalysis.ai/models/gpt-6-astra

Who is wrong here?

Some benchmark results in Astra page for Fable and Opus are blank (-).

What is Artificial Analysis intelligence index measuring that Astra scores poorly on?

Can someone from OpenAI / Artificial Analysis comment / clarify?

Even OpenAI Astra page mentions the low scope from Artificial Analysis for Astra.

parent tintor 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton