at this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents, of which I linked a couple of examples.There isn't really though. Superintelligence, while an interesting book, doesn't do much more than pontificate on sci-fi futures based largely on writing from Yudkowski. Bostrom gives no clear pathway to AGI. Neither does Yudkowski in his own writings or Barrat (Our final invention). By the way Super intelligence is kind of an offshoot book from Global Catastrophic Risks.
All of them take these interesting approaches, WBE, BCI, AGI etc... and just assume they will achieve the goal without looking realistically about where the field is or how it would get there.
So what they do is they say: We are here right now. In some possible fantasy world we could be there. The problem is they can't draw the dots between them.
For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea. What about interface, what is the interface for an AGI?
Even better (this was my thesis project): What group would have a strong enough requirement that an AGI would be required to build? Industry, Academia, Military? Ok great, can they get funding or resources for it?
etc...
Note, I am not saying that there is no potential that AGI could be a problem. The point here is that nobody has firm footing to say that it will definitely be a huge risk and that we need to regulate it's development.
There are actually people calling for legislation to prevent/slow AGI development - see Altman's blog post and many of the writings on MIRI. So that is what I am rallying against.
We need more development into AGI not less.
There isn't really though. Superintelligence, while an interesting book, doesn't do much more than pontificate on sci-fi futures based largely on writing from Yudkowski. Bostrom gives no clear pathway to AGI. Neither does Yudkowski in his own writings or Barrat (Our final invention). By the way Super intelligence is kind of an offshoot book from Global Catastrophic Risks.
All of them take these interesting approaches, WBE, BCI, AGI etc... and just assume they will achieve the goal without looking realistically about where the field is or how it would get there.
So what they do is they say: We are here right now. In some possible fantasy world we could be there. The problem is they can't draw the dots between them.
For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea. What about interface, what is the interface for an AGI?
Even better (this was my thesis project): What group would have a strong enough requirement that an AGI would be required to build? Industry, Academia, Military? Ok great, can they get funding or resources for it?
etc...
Note, I am not saying that there is no potential that AGI could be a problem. The point here is that nobody has firm footing to say that it will definitely be a huge risk and that we need to regulate it's development.
There are actually people calling for legislation to prevent/slow AGI development - see Altman's blog post and many of the writings on MIRI. So that is what I am rallying against.
We need more development into AGI not less.