>- "Paperclip maximizer". A corporation is a paperclip maximizer. If you take Milton Friedman seriously, that's what a corporation is supposed to do. As I point out occasionally, most of the ethical problems foreseen for AIs are known ethical problems with corporations.
Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a paperclip maximizer is an abstraction that obviously falls apart in the extremes, if for no other reason than because at some point destroying the world for the sake of more paperclips is going to have a negative ROI....
Meanwhile, the (AI) paperclip maximizer is actually trying to maximize paperclips and does not have the sort of internal checks that would apply to real-world orgs. That is the crux of the problem.
Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a paperclip maximizer is an abstraction that obviously falls apart in the extremes, if for no other reason than because at some point destroying the world for the sake of more paperclips is going to have a negative ROI....
Meanwhile, the (AI) paperclip maximizer is actually trying to maximize paperclips and does not have the sort of internal checks that would apply to real-world orgs. That is the crux of the problem.