"What we don't have to worry about is the AI algorithm running them, because that's speculative. There isn't an AI algorithm that's good enough to do that for the time being."
By the time a risk does hit, it will be far too late to invent the requisite mathematics necessary for machine safety. There are many paths towards dangerous AI futures and many powerful forces that could inadvertently tip the boat the same way we are escaping the implicit "goals" of our genes, so the union of the respective probabilities is very much not speculative and must be taken seriously.
"There's a whole other problem area that has to do with neuroscience, where if we pretend we understand things before we do, we do damage to science, not just because we raise expectations and then fail to meet them repeatedly, but because we confuse generations of young scientists. Just to be absolutely clear, we don't know how most kinds of thoughts are represented in the brain. We're starting to understand a little bit about some narrow things."
By the time a risk does hit, it will be far too late to invent the requisite mathematics necessary for machine safety. There are many paths towards dangerous AI futures and many powerful forces that could inadvertently tip the boat the same way we are escaping the implicit "goals" of our genes, so the union of the respective probabilities is very much not speculative and must be taken seriously.
"There's a whole other problem area that has to do with neuroscience, where if we pretend we understand things before we do, we do damage to science, not just because we raise expectations and then fail to meet them repeatedly, but because we confuse generations of young scientists. Just to be absolutely clear, we don't know how most kinds of thoughts are represented in the brain. We're starting to understand a little bit about some narrow things."
Appealing to scientific ignorance is always a bad idea: http://lesswrong.com/lw/kj/no_one_knows_what_science_doesnt_...