againstapples

Not really, social media is just surfacing the worst of a group as always. No one talks about the actual scientists like Geoffrey Hinton, Yoshua Bengio, Stuart Russel, etc who are all equally concerned about safety and have been warning about it for years now.

The same thing happens for every topic because people who are loudly wrong are more outrageous and engaging to social media users than experts, this is how you end up with people not wearing masks in the worst pandemic since the Spanish flu, things like QAnon, cancel culture, wokeness, anti-wokeness, the list goes on.

I think they mean the smarter than human kind of superintelligence not the newspeak one

What kind of things do you predict will happen?

Definitely the former, for a p(doom) I usually just say >50% if superintelligent AI is built.

I think so, individual dictators have gotten pretty far and they weren't infinitely smart. I think infinitely smart would be enough to extend that to the whole world.

> What happens when politicians can no longer lie without being caught out live on air?

If there is perfect lie detecting technology I could see all kinds of chaos resulting from it. I can't see it only be applied only to politicians, and I think it would be the developers of the technology who decide the use.

I think were we disagree is that you sort of see AI as an extension of technological progress whereas I see it more like an extension of evolution. I view the process of AI training as functioning in a similar way to evolution in that it build circuits into neural networks similar to how evolution built circuits into human brains.

It has limitations for sure, I just don't expect those limitations to last. What probability would you put on the limitations being overcome in the next 5 years?

I don't think evil mathematicians are very common or that any of them would be capable of single handedly taking over the world if they were. My concern is more about systems that are beyond human level, those kinds of systems would actually be dangerous to us.

I see the recent progress in mathematics and cybersecurity as signs that models are getting more capable more quickly than usual. The companies plans to develop them by recursive self improvement now seems like a real possibility and I don't think they should be allowed to attempt this.

I believe the AI labs might actually succeed in developing superintelligent AI and recursive self improvement, and that if they do they are very likely to lose control of the system they build.

I really think the only place people disagree is that they don't actually think it's possible, they see it as hype or doomerism. I can't find any good reasons to rule out that the companies could actually achieve what they are trying to so I think they should be stopped.

As an AI "doomer" can I ask the non-doomer people here how you interpret the significance of results like these, and what kind of progress you expect to see in the next 1-5 years?

Like do you see the technology plateauing at the current level, do you expect progress will continue but only in mathematics, I'm interested to know why others are not concerned?

What additional things would be needed to classify any future incidents as rogue?

Why can’t it have intent? How does it solve problems without intending to?

What about their beliefs is morally abhorrent?

I don’t see why I would judge a whole group based on isolated examples?

I agree they’re kinda weird but are they really that bad?