matthewdgreen

If you don't have a check in function A, but multiple functions away in the stack there is a check, I guess that could be a compiler-enforced thing. But it doesn't feel like one to me. Maybe compilers have just gotten better since I wrote C++/C code, and now that sort of pattern can be enforced.

Horrifying was the amount of memory discipline they had to use to make sure every possible message was always copied into buffers that could never overflow, and that didn’t get freed and allocated in unsafe ways. Someone poured enormous effort into enforcing those checks, which a safer language would have enforced automatically. It’s possible they now have compiler layers or tooling that does it for them, but if so those weren’t present for the last batch of CVEs. And if they do have tooling I doubt they use it on all their code because many decisions seem very specific to the buffer sizes they need for BGE parsing.

I was only a passive observer on most of this, but the feeling was “oh look, an unchecked copy, maybe this is exploitable, four layers out towards the attack surface, oh nope they have explicit checks here that lock everything down, rinse repeat N times.”

ETA: Maybe I’ll get GLM to write up its thoughts and post them here later.

I had Abliterated GLM pull one of the Core router firmwares and spent about $150 reverse engineering the BGP parsing code. It was pretty horrifying seeing all of that C/C++ emerge from Ghidra. However, it found nothing exploitable at all: the code seems to have been through so much trial by fire (including CVEs within the last few years) that they just poured engineering discipline into making it safe, at least at the level of memory management. But the cost of maintaining code like that must be enormous, and I doubt it scales to all of their other products and interfaces. I’m very much on team rewrite.

Well we haven’t quite gotten there. But models trained purely on DNA sequences have demonstrated in-context learning on various simple reasoning tasks.

Grocery stores can be as behind the times as they want. I'm worried about hospital fridges.

You have to wonder if this site will go next.

Not really plausible if it was used for all the curves.

The number of weak classes of elliptic curve would have to be gargantuan for this specific search to produce some. If that many curves are bad, then ECC was a mistake.

They somehow talk to it and it reveals this through its inability to understand complex language.

I loved that book, but Alastair Reynolds cannot write normal human beings to save his life. He's probably the worst offender.

Good characters are rare in hard sci-fi, but I do find it frustrating when (human) characters do illogical or inhuman things, just to move the plot forward. Arthur C. Clarke's characters were cardboard cutouts, but at least they didn't act like weirdos just so the humans could get aboard Rama.

A Chinese Room that isn't one.

I feel like the book cheats a little bit. On the one hand, it really wants to explore the p-zombie question, which is (roughly) "imagine there were a non-conscious being that, from the outside, could do everything a conscious one could do." Then, to drive the plot forward, it needs to posit that the non-conscious and conscious beings are actually radically different in their external actions and capabilities. At one point there's a Chinese Room that's obviously non-conscious, but the whole point of the Chinese Room hypothesis is that you can't easily tell the difference from the words it produces.

Everyone involved in this discussion understands that evolution doesn’t create things for a purpose, that it’s just the result of selection pressures. Everyone also uses the more active phrasing because it’s much more convenient. Unless you think you’re talking to someone who genuinely doesn’t understand the nature of the process, it’s not useful to be pedantic about it.

My colleagues at NIST just got a memo directing them to refer to AI as “SI” so it does seem to be filtering down to the agencies. This is a problem because “SI” is also a classification marking.