lordofgibbons

I dug into it more, and it seems like the reasoning efforts are available for the Intelligence Index, but not the Coding Agents benchmark I was interested in.

I see them for GPT 6.1 Terra, but not Opus 5.5. I'm looking on the Coding Agents page.

I wish AA benchmarked the models at the reasoning efforts people actual use them at. For example, it's very rare for people that I know to use the "max" reasoning setting - specially because Claude Code itself scares you away from using it. But all of these coding benchmarks are based on just that reasoning effort.

I get it, they can only benchmark so much. But I'd rather have a benchmark of how the model is actually used and offered to customers rather than the best case scenario for it.

It's similar to the ElectronJS desktop app problem. Sure the desktop apps use a lot more system resources, but it's doubtful those apps would have been created in the first place if it wasn't for the speedup of development allowed by ElectronJS.

In this case, sure software is buggier, but there's also a ton more software+features that wouldn't have existed before AI coding tools.

The difference is that releasing the model for free doesn't have ongoing cost for the company. Providing cheap tokens is very expensive - specially if you don't have access to the latest transistor node chips. So I think the parent comment is right, there's something else at play allowing DS and Xiaomi to offer these nearly free tokens.

What's the verifiable oracle in this scenario?

Without showing false-positive rates this analysis is useless.

If your model says every line if your code has a bug, it will catch 100% of the bugs, but it's not useful at all. They tested false-positives with only a single bug...

I'm not defending anthropic and openai either. Their numbers are garbage too since they don't produce false-positive rates either.

Why is this "analysis" making the rounds?

Given that Google has said they'll be delaying source code release for Android to every X months intervals (iirc), how is GrapheneOS planning to handle security updates? Will they just be Google's binary blobs?

The controversy isn't about defending one's country, it's about you and the parent comment author assuming what this is all about without reading the article.

The core of the issue about autonomous use of AI in mass surveillance of Americans and autonomous use of AI in automated weapons that make kill decisions. Anthropic is perfectly fine with working with the War Department and "defending one's nation".

But they are not okay with their AI being used to make a mockery of the 4th amendment and making automated kill/no-kill decisions about actual human lives.

Pardon my ignorance, but doesn't a byte-by-byte output recreation of the C++ code in Rust defeat the whole purpose of using Rust? For one, would it be idiomatic Rust anymore? Also, if there's a (non-memory related) vulnerability in the C++ code, would it be possible for that to be introduced in Rust too?

This looks great! I have a couple of questions:

1) Is it possible to start off with plain Postgres and add pgdog without scheduled downtime down the road when scaling via sharding becomes necessary?

2) How are schema updates handled when using physical multi-tenancy? Does pgdog just loop over all the databases that it knows about and issues the update schema command to each?

I'm currently using Ghostty with Zellij, and there has been a constant tension w.r.t whether I should use a zellij feature or a Ghostty one (i.e, tabs/panes/etc) when they provide the same thing.

I've come to the conclusion to rely more on Zellij because I can SSH into my desktop from my laptop remotely to continue my dev session exactly where I left off.

So, these days I don't even use "native" terminal tabs anymore.

You can try raising the head part of the bed by 5 - 6 or so inches using wood blocks. The doctor recommended it to me.

It's not perfect, but has really helped me!

How interesting it is depends purely on your use-case. For me this is the perfect size for running fine-tuning experiments.

5 years used to be true 15 years ago. IPOs are happening much later now, probably 10+ years, if you're lucky