rogers18445
Joined 5 years ago
Karma
348
Submissions
193
Member since
Apr 5
Joined 5 years ago
The accidents were spectacular, even though they resulted in so very few deaths. For most people nuclear is effectively witchcraft, and selling fear is effective. The HBO series were so absurd yet so loved.
Ultimately, the reward for attaining an AGI agent is so high, that no matter the penalty someone will attempt it, and someone will eventually succeed. And that likelihood will ensure everyone will want to attempt it.
It's not at all obvious that a super AI would have an intellect incomprehensible to a human the same way a human is to a chimpanzee.
Or another way to phrase it, it's not at all obvious that an AI can exist that would be incomprehensible to a very smart human, it may however reason much faster than such a human.