Comment by nytesky

I don’t see any positive outcome if we reach AGI.

1) we have engineered a sentient being but built it to want to be our slave; how is that moral

2) same start, but instead of it wanting to serve us, we keep it entrappped. Which this article suggests is long term impossible

3) we create agi and let them run free and hope for cooperation, but as Neanderthals we must realize we are competing for same limited resources

Of course, you can further counter that by stopping, we have prevented the formation of their existence, which is a different moral dilemma.

Honestly, i feel we should step back and understand human intelligence better and reflect on that before proceeding