bermudi

Impossible to even know if he was talking about the harness, the model, the assistant... But yeah, sticky... Ok.

Muse 1.2 wrote a terrible "smart summaries" extension for my pi setup. It was sending every single steamed chunk for summarization instead of waiting for the full CMD.

This is an error I would expect from sonnet 4, not a model that was supposedly just a few points behind sol.

I honestly can't believe serious people are making this argument on a straight face.

Gemini 3.7 flash outputs so many tokens per answer it doesn't matter how fast its TPS is, sol will end up being both cheaper and faster than Gemini. So ppl are paying more for a given task, waiting longer and using a dumber intelligence because "TPS number shiny".

Gemini 3.8 outputs 11k more tokens PER TASK on average in AAII than 3.7 putting it dead last in output tokens per task in the leaderboard.

This only makes me understand how flawed AAII is. This Qwen model is nowhere close to the other models in that score range.

If you're working on smol? How complex is your work? The agent has to do everything using sed? Did you write your own tools? I guess my question is, why aren't you using pi?

The difference in tokens between the two also makes super curious. The system prompt can't be that different (I'd even bet Pi's shorter) and the 4 tools shouldn't make as much of a difference. I'm gonna have to try it.

DeepSeek being DeepSeek. v3 and R1 went over the same and had multiple versions

Source? The most trusted benchmark right now (deepSWE) scores better or just as well on their minimal harness than when using CC or codex

I wonder if you're as cynical and untrustworthy of American companies as well or is it more of a racism kinda thing

Everything is more expensive than deepseek. They aren't frontier in intelligence but they are the frontier in cost per intelligence

What surprised me the most was the fact that people have enough disposable income to pay for search to make this viable