chakalakasp

Because encryption is not an inherently dangerous thing. A superintelligent AI is.

It’s no different than inviting an advanced alien species to visit. Will it go well? Sure hope so, because if they don’t want it to go well it won’t be our planet any more

You have no clue what communities I’m in other than it self affirms your worldview to assume that they must be irrelevant. To be fair, I’m not telling you anything about myself, so you’re not really needing to take my word for it. And I don’t care enough about you to explain in detail.

Though with a tiny bit of Googling you’ll be able to find several Turing Award winners who are saying exactly what I’m saying. In public. Loudly.

The difference is the scale at which a hacker can cause damage. A hacker can ruin a lot of stuff but is unlikely to kill a billion or two people if he succeeds at a hack.

With superintelligent AI you likely have to have every developed use and every end user get it right, air tight, every time, forever.

In what other sphere of tech does that apply?

"Hackers aren't a problem because we have cybersecurity engineers". And yet somehow entire enterprises and governments are occasionally taken down.

What prevents issues in redteam/blueteam is having teams invested in the survivability of the people their organization is working for. That breaks down a bit when all it takes is one biomedical researcher whose wife just left him to have an AI help him craft a society ending infectious agent. Force multipliers are somewhat tempered when put in the hands of governments but not so much with individuals. Which is why people in some countries are allowed to have firearms and in some countries are not, but in no countries are individuals allowed to legally possess or manufacture WMDs. Because if everyone can have equal and easy access to WMDs, advanced civilization ends.

If you talk to the people on the bleeding edge of AI research instead of nuke-heads (I tend to be kinda deep into both communities), you'll get a better picture that, yeah, a lot of people who work on AI really do think that AI is like nukes in the scale of force multiplication it will be capable of in the near to medium future, and may well vastly exceed nukes in this regard. Even in the "good" scenarios you're looking at a future where people with relatively small resources will have access to information that would create disruptive offensive capabilities, be it biological or technological or whatever. In worse scenarios, people aren't even in the picture any more, the AIs are just working with or fighting each other and we are in the way.

I dunno. The American Geophysical Union seemed to think it was a BFD in one of their conferences back in the 2000s. Had a whole presentation about how certain parts of the US grid would lose large transformers that would likely take months to years to replace as there are few spares and they have to be wound with miles and miles of copper by hand. Said that large swaths of the US would be without grid power for very long periods.

Said that third world countries would fare pretty well though.

This event is twice as powerful as the previously known largest event, which is at least 10X as powerful as the largest event in recorded human history, back in the 1800s, which set telegraph poles on fire. So, yeah, it's a big Twinkee.

Such as the humans

This is a succinct explanation of the problem. Do we give the vast majority of users extremely easy, frictionless access to very high levels of security and privacy? Or do we give the vast majority of users a fundamentally insecure solution that with lots of learning and configuring and time can be have very very very high levels of security and privacy?

The crazy thing is that apple hardware beats most other hardware, too, at a high price. Better phones, better tablets, better laptops. More secure, more private OS than the popular consumer alternatives (Windows, Android). Arguably much better OS all around, too (at least IMO -- iOS beats even stock Pixel Android at use-ability, MacOS v Windows is like the Harlem Globetrotters playing the Washington Generals.)

Thing is, if law enforcement is patient they can get the data off the actual devices themselves, if they're still alive. Yes, a fully patched iPhone tends to be a fortress of might to anyone other than a nation state willing to burn a few very expensive 0 days, but with almost any phone if you wait a year or two something will inevitably come out that will allow the ol' Cellebrite crowbar a cranny to slip into.

My favorite part of android is how security patches go through a multi-tiered trickle-down system of testing to make sure they work with the dozens of custom flavors each manufacturer has so that by the time you get patched it's been in the wild for weeks or months. Oooh, ooh, no that's not my favorite thing, my favorite thing is how each cellular company gets to put their own bloatware on top of the bloatware that each phone manufacturer gets to add to it. Oh wait, maybe it's patch support ending for new phones 3 years after they were released. There is so much to love about how Android turned out it's hard to pick just one thing.

Except, of course, the human has very little control over what the AI outputs in TXT2TXT scenarios, at least in terms of whether the output would match the definition of copyright infringement of someone else's work. IMG2TXT is kinda different -- I think you could make a much stronger case for derivative work there. So you have a tool that can randomly create massive liability for you, and you can't know if its done so or not until someone sues you.