AdamConwayIE

Hey, article author here. No, it's not an AI article. I've been sick the past couple of days and overlooked that heading. What I meant by it was that peripheral support changes how the device positions itself compared to others on the market.

I'll fix it, and thanks for mentioning it, but I also wanted to add that the "AI article" accusation was unnecessary. I spent a lot of time going through Espressif's documentation and the datasheet, and working out how Sv32 works and differs from the "MMU" implementation Espressif typically touts. It just feels like a strange accusation to tack on to an otherwise fair point regarding the article's readability.

Not sure to be honest. I have a couple of CVEs and also linked a talk I had presented as it mentioned those as an example. Got approved in a few minutes.

Maybe so, but there were other elements that I've seen frontier models struggle with in the past, which was the perspective I had coming into this. It's the type of test I run frequently and this is the first small local model I've seen pull it off.

It had a very non-standard RSA key implementation that was obfuscated heavily. As well, it has an online license check at first run, and that part typically trips up most of the local models I've tried. I've been running this test for about a year now with different models, and it was the first I've seen not only figure out the RSA key implementation, but the first that didn't just give up once it saw the online license check. Even though it's only a first-time launch check.

That's why I call it one of the hardest, because in my experience, it has been. It's the first local model I've seen pull it off end-to-end. For some of the reverse engineering work that I've done with LLMs, none have been as consistent as this particular test at highlighting a model's failure in this domain.

I have access to Daybreak Blue and I'm approved for Anthropic's Cybersecurity program, so I might run the same test with both of those just to see, because it's been a while since I used a frontier model on this test. I imagine they'll make relatively light work of it, though, assuming it doesn't trip the relaxed guardrails.

I added a line to address this, sorry it wasn't there before! It was Pi and only used Bash-based tools.

Nope, it wasn't.

Ah, my bad! This image came from our backend, used for an unrelated article. I selected it by mistake rather than inserting the actual image that I'd uploaded. I'm updating it, thanks for the heads up!

For what it's worth, that image couldn't have been related. The other screenshots all showed thinking traces, and Claude doesn't share those.

Oh wow! Article writer here, can't believe I didn't come across this when I was researching. Kicking myself haha

Yeah, I'm reminded of container escapes, VM escapes etc. There have been plenty in the past; VirtualBox E1000 (I think?) comes to mind from a few years ago. If we are to believe this model can find 0days, I'm on board with the idea it could do so in sandbox.

That's not to say I believe it outright, but people are being oddly dismissive and acting as if it's impossible to break out of a sandbox. Which we've seen time and time again that it absolutely can be.

I've been loving LiveView. Been using it for a project for a client recently and it's so... chill. I like it a lot.

I remember using Orb on the Wii way back then for media streaming. Good times!

Had a very similar experience recently.

Built a basic authentication handler for this test just so it wouldn't be in the training data of either model. It had deliberately planted bugs. One was a hardcoded secret, another was a wrap-on-0xFFFFFFFF bug as a result of a malloc(length+1).

Qwen 3.6 found both, alongside two other issues I hadn't even considered, and the location of the magic value. GPT-5.4, though, missed the malloc issue (flagging memory exhaustion as the only risk), it missed a separate timing bug (it explicitly said the function was safe), and it hallucinated the location of the magic value. Qwen correctly identified the integer overflow. GPT-5.4 did not.

I then compared basic research between them using SearXNG for web search. For example, the current status of MTP in llama.cpp. Qwen 3.6 27B found the current PR, but flagged a related issue that shows the current implementation can be slower than just using a draft model right now. GPT-5.5 Thinking found the same PR, but didn't flag the downsides.

In a similar comparison, I asked both models how I should get started with ESPHome as a total beginner. ChatGPT suggested an ESP32-S3 and a BME280, which is... just not a good idea. It also talked about the ESP32-P4 not having Wi-Fi, and installing with HA or Docker. Meanwhile, Qwen3.6 27B said regular ESP32, DHT22, and mentioned HA, Docker, and pip as installation methods. While GPT was good, it was just throwing out jargon for a prompt that explicitly requested it for a beginner.

It kind of blew my mind that in all three of these, Qwen landed it better.

Hey, article author here!

I've been writing for nearly a decade, and I can assure you, all of this is human written. I've long been writing about the Linux kernel where it's been relevant to my coverage, and there are articles under my name talking about low-level technical aspects in drivers and kernels from as far back as 2017.

I get that it's hard to know what to trust out there given that Dead Internet Theory is beginning to feel like a reality, but comments like this can be quite upsetting after spending days researching and writing an article like this. I totally get criticism of the article itself, and I'm fine with that, but it feels as if people are too quick to jump on the "must be written by AI" bandwagon. I receive it, my colleagues receive it, and for the people who I know put in so much effort into their work, it can be upsetting to them as well.

As was mentioned in another thread, there were actually a couple of typos in this article when it went live. I cleaned those up once they were pointed out, but AI doesn't make typos. I get it to an extent; hostility and accusations of all kinds have been levied at writers for the years and years I've been in this industry writing long-form content and analysis. But with the proliferation of AI, that hostility has really ramped up over the last couple of years.

There aren't really any of the typical benchmark suites targeting Codex 5.3 because it's still not in the API.

SWE bench for example creates a predictions file and evaluates the results in the harness. Without Codex 5.3 being in the API, it can't.