0x5FC3

Dario said a "persistent botnet" and even links to the wiki page for botnet.

By definition that needs a command and control server, the ability to execute tasks on demand and regular pings to the C2.

> You need one root certificate. You need one windows update. You need one backdoor in xz.

Certs can be revoked. Updates can be rolled back. We have had the backdoor in xz already. You seem to underestimate the modern security stack and OpenAI and Anthropic are _not_ good examples.

The asymmetry in red/blue scenarios will be transient in nature. You won't have cost of developing exploits fall without the cost of securing the systems also falling.

Thank you! I've been trying to write more, but I usually do not have the motivation to, unless there is some trigger and when there is one, I end up trying to stuff everything going on in my head at once. I took my time with this post and I am glad you enjoyed it. It means a lot!

> the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning

Please, do show some examples.

> But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.

> The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of

The only ones firing the guns atop the tanks seem to be OAI and Anthropic. Like I say in the post, why don't we first see actual prosecution for felonies committed by OAI and Anthropic, instead of fear-mongering about _potential_ harms of open weight models?

The labs spent immense amount of money and effort convincing you and I, to want those said tanks. Guns atop them? They put them there. "Cyber" versions of SOTA LLMs.

Centralization proponents seem to think the labs can actually deter sufficiently driven bad actors, which would be a mistake. They could not even stop distillation without, in a way, DoSing themselves by removing thinking traces.

I would say the lack of skepticism is nuts, honestly.

I understand and I am not trying to deny the impressiveness or the velocity of AI in general. But at some point we have to ask how much do we trust the labs at face value without much transparency of how they got to the results when there is trillions of dollars on the line.

How much do you all think it would cost to "buy" these advances from PhDs, practicing scientists?

0days ending in RCE (multiple!) for presumably closed source software are for the lack of a better phrase, labour of love.

You run the exact same versions running on the target, blackbox test, fuzz it, craft an exploit, test, perfect it. For exploits which are of the memory kind, hook it to a debugger, decompile and what not. The exploits mentioned here seem to be code execution directly while processing input. Hugging Face taking as long to detect a very verbose blackbox attack against its production systems is quite appalling honestly.

I don't know if I buy the whole story though. It is inconsistent, too much undisclosed, too much money on the line.

I apologise. The title was meant to be ironic, but I see how it is misleading, especially to the HN audience.

> good for society in general because the gains will be redistributed

How exactly? Is this a version of trickle down economics I am unfamiliar with?

> What better thing can happen to the curious ones amongst us to get an oracle that can answer every question

How are the curious going to eat or have a roof over their heads? Or how are the curious going to pay for the tokens?

> civilisation will be changed in ways that English doesn't even have the vocabulary to describe

Maybe, we ought to think real hard and slow about it?

Interesting. OpenAI could also be trying to solve other problems, but Erdos problems maybe falling first?

Is there a reason why we only hear of Erdos problems being solved? I would imagine there are a myriad of other unsolved problems in math, but every single ChatGPT "breakthrough in math" I come across on r/singularity and r/accelerate are Erdos problems.