Comment by lanfeust6

Similar sentiment, have not read the book but I imagine I've absorbed some of it through rat/ea circles and here. I can appreciate an argument that rapidly-improving AI, with mere probabilistic output, could lead to bad outcomes owing to neglect and complacency, or aiding at wreaking havoc. What I don't buy is the argument that sentient super-intelligence could "accidentally" manifest itself through recursive self-improvement, which is along few dimensions. I'm not sure it would be even possible to intentionally create one, but it would be redundant; you don't need RSI to generate the value people want from AI. We could pass strict laws against that, but it wouldn't satisfy the doomers who are already arguing that human minds are "just an algorithm", as though AI sentience is a foregone conclusion