adriand

We are not helpless. We can choose to control the technology and do so in a way that is in line with democratic principles. After Hiroshima, people assumed that most nations would develop nuclear weapons and that nuclear war would be commonplace. But Nagasaki was the last nuclear attack, and 80 years on, only a dozen nations have nukes. This technology makes nuclear weapons look like toys. We either choose to control it or we will be the victims of it.

Are these cyberattackers using models they developed themselves? Or are they using models they stripped safeguards from? It’s the latter, right? And it’s only going to get worse if we don’t take steps to control the technology.

Maybe Bill Gates has a point when he says that models capable of significant cyberattacks need to run on controlled infrastructure. Is the era of open weight models you can strip safeguards from turning out to be so great? From my perspective, this kind of sucks.

> Cursor should not be shipping extensions that are branded Anthropic if Anthropic didn’t create that extension.

Well, duh. From what I can tell, that’s literally the hack. A bad actor shipped an extension on the Cursor marketplace that wasn’t made by Anthropic.

I’m of two minds on utopias. On the one hand it seems we need a positive vision for the future, where technology is used to solve our greatest problems and improve the planet. On the other hand it seems that perhaps technology is incapable of making us happier, because it isn’t clear to me that we are happier now than in past decades, despite having more advanced technology.

The deniers have transitioned to doomers. Originally they said we needn’t fight climate change because it wasn’t real. Now that’s it’s undeniable they say we can’t fight climate change because it’s too late.

> Try to find insightful reflection of humanity and truth in a Marvel movie... God help you. It's as boring as, well, watching someone else play a video game.

It’s so true. The only time I watch one of these movies is when I’m stressed and exhausted and want something so dumb I’ll fall asleep in 20 minutes.

When it comes to these “worlds” that are generated, there’s no there there. I’ve tried various of these and after three minutes I close the browser. Walking around my neighborhood is a million times more interesting. Whereas the creator-constrained and -designed experiences (GTA comes to mind) actually deliver something engaging.

> Both robots are magnetic and guided wirelessly by an external electromagnetic coil system

That’s so cool. It reminds me of being a kid and playing with magnets beneath a sheet of cardboard, using them to invisibly move metal objects on the surface. Is that sort of the principle here?

> After the LK-99 debacle

"Debacle"? That was the most fun I've had on the Internet in years. When's the last time so many people engaged in so many arguments about materials science and electromagnetism? Sometime in the 1800s?

We don’t need to regulate “computation”, we need to regulate artificial intelligence. The “proposal” you outlined is a straw man, I’m not opposed to open research, and I have no idea who you are referring to by “our former employees”. But yes, we should be regulating this technology like we regulate nuclear technology.

Jensen Huang does not want regulation, but in his interview with Ezra Klein he said AI will let us “know anything” and “do anything”. Do we actually want any random person to be able to “do anything”? I remember when the Japanese doomsday cult Aum Shinrikyo attacked the Tokyo subway with sarin. Do we want doomsday cults to be able to “know anything” and “do anything” so that instead of releasing sarin, they release a genetically modified strain of smallpox?

This is not a hypothetical risk, this is an actual, present danger. And good luck trying to vaccinate yourself against an engineered superflu using a Chinese open weight model.

This completely ignores the world we are creating here. We are setting ourselves up for a massive cyber disaster. The safeguards on US models are insufficient but at least there are safeguards. The fact there are open weight models floating around that are capable of wrecking the economy is a genuine problem! One that Anthropic is doing us a service by warning us about.

When someone wielding a non-safeguarded model deletes the money in everyone’s bank account, I look forward to the HN comments claiming it’s an attempt by Anthropic to pull off regulatory capture.

Well, I've been sitting on Sequoia this whole time, because I decided to sit Tahoe out, based on what I heard on HN. Maybe I'll sit Golden Gate out too!