boutell

But with that version number, it's not a "minor" update! It's a patchlevel update! -Engineer

The big players tend to leave out each other's very absolute latest models too. There's a nice ambiguity there: was it genuinely too new to be included when they made this? That ambiguity can do a lot of work.

For what it's worth, they claim they applied "large-scale reinforcement learning" to a base of glm 5.3. If that's true, then it's not as trivial as a lot of the "startups" have been lately.

I understand what you're saying. My first thought is "is that really applicable to a mistake like the text on the T-shirt extending over the suit jacket that partially covers it?" Maybe it is. I also don't know what you actually get when you prompt Krea 3 via their website as I did today. I don't know whether that folds in some guidance from an LLM first. Tough to say.

Gave it a photo of a woman wearing a T-shirt, asked it to seat her at a fancy bar. It invented a suit jacket, which makes sense, but the drawing on the shirt runs right over the jacket. Gemini did much better.

It’s pretty damn hard to use on a desktop! I was rolling the mouse like it was a manual steering wheel

Recently, I’ve discovered that Claude can render great SVG versions of user interface screenshots. I ask it to substitute simple SVG friendly landscapes for any photographic images that happen to be part of the screenshots. The results are much better than they should be, especially with Opus. Usually good enough to be used on a website as a way to get an infinitely scalable, much smaller version of a screenshot. But clearly, I should try the Gemini models on this real world task. That’s where we’ll find out if all of this training to beat the benchmark actually pays off.

The previous, human-discovered result is the same to... how many decimal places? It wasn't printed in enough detail on the site I found to even say...

Still an achievement, just funny.

Yes, and Qwen 3.8 27b is a similar inflection point for local, although I wouldn't claim it's quite as good as Opus 4.5 it is genuinely useful. Whether that actually matters will depend on whether it ever becomes the most cost-effective tool for the job, but it's impressive as all heck

Absolutely possible, which is why I refuse to take my eyes off code review (mine, or a few other trusted souls)

I'll speak up in defense of the reporters: FWIW, so far my strong impression is we're hearing from independent security researchers. Right now, for us, so far (enough qualifiers yet?) the system is working for us: independent security researchers are farming reputation by finding real problems. That's not a bad thing.

And in most cases they do propose solutions, although we generally resolve the issues on our own.

There's a small percentage where we make the case that the ticket is not a real vulnerability, and then we have to grit our teeth through repeated reports of the same "vulnerability." But it's a small percentage so far.

We do typically have to reconsider the severity. The researchers understandably want to see everything as a nine...

I head a much, much smaller open source project. Since the November Singularity we've been seeing at least six responsibly reported security advisories a month. However, this last month we had 22 unique security advisories. Our project has been built with adherence to the OWASP Top Ten Guidelines and other best practices from the beginning. But software is hard and AI is thorough.

Each month, we fix them all in our monthly maintenance release and disclose at that time. We fight AI fire with fire, and hand-review, of course.

So far, we can keep up. One hopes this is possible at the scale of the Linux project, which assuredly has more humans and more AI to throw at the problem. But team size does not scale linearly with interested audience, and potential bugs do scale with codebase size (and other extremely important factors, like code quality, at which the Linux team is assuredly much better than we are).

("November Singularity" is a cheeky reference to the arrival of Opus 4.5 and "good enough" coding models and harnesses generally.)

I know this is just one tiny piece of the omnishambles, but: can't github sidestep the "you have to tell us what kind of hashes you have at repo creation time" by effectively deferring "git init" in practice until it sees what's being pushed? There may well be a good reason this wouldn't work...