alphabettsy

Isn’t Kimi k3 more expensive than Opus 5.5 in the benchmarks?

Of course, benchmarks don’t tell you how much it will cost on your workload.

GLM 5.3 was more expensive than Haiku and Sonnet 5.5 on a workload I was testing recently despite being much cheaper on the price sheet.

I would love for this to be true, but this is nonsense. The local models that you can run for those prices are not as capable as current frontier. At $200 a month, it would take well over a year to break even, and that’s if you don't include energy prices and time spent tinkering to get and keep it working.

On some benchmarks models like Qwen 3.8 Max which cost < $6/m out cost more than Astra 6 to run at $50/m out. That’s a huge price gap and yet Astra would be cheaper if your work looks like the benchmark.

It might be objectively true for you.

Opus 5 is fine for me and works better and faster on low and medium than higher effort on prior versions. Same as 5.6 Sol compared to 5.5 or 5.4.

A total measured over the entire US economy without adjusting for a variety of factors including inflation and population change seems like exactly the kind of thing you would expect from a news headline. How about average or per capita?

As always, it requires evaluation with your work because I’m often finding grok to be much more expensive than the price would lead you to believe.

There’s also the frustration of it not quite being enough sometimes. It’s extremely capable, but I still find that it needs more concrete guidance and boundaries than other models.

GitHub has been providing macOS runners for a while now.

That’s unfortunate. I’m not interested in using Musk associated products anymore than I have to.

I’m trying to understand how this is useful information on its own?

Maybe I missed it, but it doesn’t tell you if it’s more successful for less overall cost?

I can easily make Sonnet 4.6 cost way more than any Opus model because while it’s cheaper per prompt it might take 10x more rounds (or never) solve a problem.

The docs suggest not using max effort in most cases to avoid overthinking :shrug:

That’s been there for awhile.

IMO it doesn’t handily beat it.

It’s typically equivalent, sometimes better, sometimes behind. Better at following a well defined plan, less good at concept exploration and planning imo.

At 1m context it’s basically the same price.

Prompt cache expired?