> If I sit down to play chess with Magnus Carlsen I can't predict the exact moves he'll use to defeat me, but that's a bad reason to think he won't defeat me
This is a very bad analogy because chess isn't life. In chess, you aren't allowed to do whatever you want. There are rules. I know for a fact that Magnus Carlsen won't beat me using checkers moves and he won't beat me by pulling out a gun and telling me to resign. Magnus Carlsen's skill at chess leading to his victory in chess is not a valid analogy here, because there's no law of nature that says "the more intelligent entity wins in a battle for survival".
You could have infinite superintelligence and still die inside a locked room to which you have no key. "Superintelligence" is not a magic solution to every problem, you can constrain any superintelligence with any unsolvable problem.
This is a very bad analogy because chess isn't life. In chess, you aren't allowed to do whatever you want. There are rules. I know for a fact that Magnus Carlsen won't beat me using checkers moves and he won't beat me by pulling out a gun and telling me to resign. Magnus Carlsen's skill at chess leading to his victory in chess is not a valid analogy here, because there's no law of nature that says "the more intelligent entity wins in a battle for survival".
You could have infinite superintelligence and still die inside a locked room to which you have no key. "Superintelligence" is not a magic solution to every problem, you can constrain any superintelligence with any unsolvable problem.