stalfie

I edited my comment above before realizing this is what you meant. And its a valid point I concede. Still, I disagree that advantaging this activity is the only thing these embargoes achieve. I would argue that they legitimately also give the Linux kernel, cURL and so on an advantage. And as long as that is true, this still makes it an overall win in my books, even if it also enables officially sanctioned offensive activities. I'm not a US citizen, but I would still rather get pwned by the CIA than both the CIA and Shinyhunters.

Well, seeing as Claude literally snitches on everyday people planning murder and Houthis building rockets I would assume that every user in project Glasswing is heavily and very effectively monitored. I have to admit that I genuinely do not think that what you're hypothesizing is happening right now. I would bet good money that the amount of people with Glasswing access doing actual Blackhat work is currently zero.

Perhaps in the future this will change, but even if that is the case, that's still a win for the defenders. Having a time advantage, even for a few months, is still a win.

Edit: Actually, the CIA/FBI/NSA/Etc.. does probably have access and are probably trying to hack Iran and so forth, so my comment above is probably technically wrong in that respect.

Ah, now that's an interesting point. However, wouldn't those groups just fine tune Kimi K3.1 pro no matter if it's nerfed or not, leaving you in the same position as before?

That is, unless the nerf is easier to correct as compared to "upgrading" the models offensive capabilities. So a scenario where it is easy to "restore" offensive capabilities, but hard to "enhance" them beyond the labs best attempt. So is that the world we're living in?

Privileged attackers?? So that would be career cybersecurity experts with Project Glasswing/Daybreak/Googlewhatever access, using it to perform actual attacks? Because that's what privileged means in this context. Access to frontier models not available to the public.

Noted. I also did it intentionally to sound a bit spicy, which seemed like a good idea at the time. So I am guilty, and I regret my actions.

I wrote a few lengthier comments on some other similar replies. Summarized my argument is that it seems to me to be strictly better that select defenders have an advantage, rather than everyone being equalized. If you're not one of the privileged few that might be frustrating, but open source models upgrade the attack and defense by the same amount, so open source cyber capabilities being at the frontier only results in a defender disadvantage.

Replying here also as I'm curious about the counterargument.

I'm not sure I follow. Let's say that GLM remains the frontier of open source cyber capabilities because Kimi is nerfed. Your situation, and the situation of your attackers, remain exactly the same. You run GLM to audit your code, they do the same looking for exploits.

Let us say that un-nerfed Kimi was released, and it turns out to be at the absolute frontier of cyber capabilities, Mythos 5.5 level, a step ahead of anything else. What now? Well now your situation is exactly the same as before, just that auditing and hacking got more expensive. You run a Kimi security audit, they do the same to try to pwn you. However, now the cabal of Linux kernel defenders with secret access to Mythos 5.5 all of a sudden have no advantage, where before they used to have one. And even worse now they are in a literal race to be the first ones to find every bug that Kimi is capable of finding, and patching it yesterday.

So for you, this makes no difference in practice, your capabilities are not elevated, and neither are those of the attackers. The only result of the other scenario is an overall defender disadvantage.

Well, and I am genuinely curious here, wouldn't defender and attacker capabilities being accessible equally by both parties be objectively worse than having a limited amount of privileged defenders? Of course, you could have every piece of code go through an adversarial audit before it's pushed, but in an equal world you would have to always devote as much or more resources to this as your attackers. And historically speaking people just don't do this in practice.

In the scenario where only "verified" defenders get access to the best cyber capabilities, unverified plebian defenders and blackhats are on the same level exactly as described above, there's just also a few defenders that get to run things on godmode.

So to me, it seems that the only result of frontier cyber capabilities is removing a defender advantage. Although that advantage is not for everyone, it can only exists as long as capabilities are unequally distributed. Am I missing something?

Well, in the publication they claim to have identified specific cybersecurity post training tasks that nerfed capabilities when they removed them, which seems like a reasonable methodology.

Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.

So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".

Well, I agree with publishing metrics being detrimental! On your other points, you're arguing against a strawman. I didn't say anything about life just being a snatch and grab, nor do I hold this belief. But university admissions, job interviews, buying non-plentiful resources...all of these are competitions. Competition permeates the very foundation of our education and financial systems. Many of our greatest virtues as a species are not competition based, but it is a large part of the reality we live in, for better or worse.

And about what happens in a society where constituents have nothing to contribute... I have no idea. But we might find out within a decade or so. The fact that what happens might not all be great is unlikely to stop any of the AI labs.

Careers in the real world are always about winning competitions, no matter what the field of mathematics is supposed to be. If not about competing for tenure, then it's a competition to publish.

And if AI can also make use of the math, then the AI is all you need in order to grasp a better tomorrow. If AI poops out a warp drive, you don't need to understand it to use it. The argument about humans needing to understand anything only holds true as long as humans have something unique to contribute, which might not be the case even a year or two from now.

If the majority of society is simply wrong as a matter of fact, should we not talk about it?

For instance, should we not talk about MMR vaccine skeptics or climate change denial in societies where they are prevalent? When the majority of a society believes something that is wrong, that is the time we should talk about it the most.

I'm gonna go ahead and make the prediction that this eventually morphs into a kind of religion, like flat earthers.

Nothing that AI will ever be valuable, everything AI does is steal, nothing it creates is new, everything that the big labs publishes is a scam or hype. Oh and "it's just doing X", where you can substitute "X" with some kind of surface level description of how the output layer in an LLM works.

These tenets are already pretty much formed, all it needs is to be formalized by some kind of name or manifesto similar to Q-anon, and we've arrived at a durable state of permanent denial.

Eh, decompiling is happening now, at current LLM capabilities. The resulting codebase compiles into something that is functionally identical to the original. What engineering principles exactly do you need beyond "works exactly the same as the original product", in this particular context?