So much of these discussions are overly focussed on an general purpose AI singularity, and what the AI itself would do.
That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do.
For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry.
The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans.
The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.
That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do.
For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry.
The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans.
The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.