oh_no

> this full release warrants a major reassessment.

I really don't think it does, the benchmark numbers are dreadful and the cost is higher than Sol. If you have to use an EU model knock yourself out but there are a ton of models that are a lot smarter for a lot less money.

based on Anthropic's S1 no one is serving models at a loss, but cutting your price to keep usage high makes a lot of sense when you have your inference booked for long periods.

AA has limited 1017 benchmarks so some terras versus 1017 Max: max - 3 points smarter, 50% more expensive xhigh - same score, 25% cheaper high - 1 point dumber, 50% cheaper

I kind of expect them to go right to Astra 6.2, 6.1 was cancelled over the cybersecurity hubbub. I eagerly await either one.

the only confusing bit was dropping terra for a bit,

Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)

is perfectly clear, sol beats astra sometimes because Astra 6.1 was scrapped. and note that opus 5.5 beats fable 5.1 most of the time too without a skipped model.

yes but going back to 2019 staffing with 2026 revenue is a pretty seismic shift

sol 6.1 is wildly cheaper, one reason to use something else

day 1 benchmarks had that for opus and i think also astra but updated numbers have smoothed that out. i can't say for sure what all changed between early runs and now but it seems like max is now the most powerful mode for both models.

that said, with a clock on this benchmark it's very possible that lower reasoning = more iteration = better code.

please be serious

also a mess on desktop firefox

i mean we use max benchmarks because they have the best coverage, which sucks because very few people use max day to day, but it's what we have. the performance curve is generally pretty similar across models and effort levels, weird outliers are pretty weird. max is generally a big cost bump from most providers (less so from OpenAI)

AA shows Luna at 1/4 the price, 1 point behind on intelligence matrix with a 38.

Haiku 5.5 is 23% cheaper with a 4 point intelligence lead.

I'm on subscription usage so I can't compare Flash 4.1 to them directly but the OP has his head up his ass if he thinks Opus 5.5 is the best point of comparison. Why is anyone using Opus if the new Haiku is indistinguishable /s

Just absolutely terrible post, admits to using Opus for review but claims its intelligence isn't needed, why aren't you using Haiku or Sonnet then?

Luna is 1 point being on AA's index at 1/4 the cost, yes it "doesn't score as well" but paying 4x for 1 point is crazy if you're going off benchmarks.

AA has Haiku 5.5 as cheaper than 4.1 Flash (both on Max, which isn't ideal but what can ya do) and a 4 point intelligence gap.

Why do people like to think open models are more competitive than they are?

then just ask for text-only?

no AA benchmarks yet and the last chart in the announcement makes Haiku look useless vs new Sonnet pricing, interesting to see what 3rd party benchmarks show because i think Anthropic are costpertaskmaxxing here and it's going to look more like that bottom chart than the top ones.