When these AI companies speak of regulation, they are considering two categories "dangers". These "dangers" are existential threats to the AI companies and the people running them, but they're much less of a threat to me, a random human individual. I am more afraid of the continued existence of companies like Anthropic, OpenAI, Google, and Microsoft, than I am afraid of the following "dangers" they want to "regulate" (via regulatory capture):
1) People will use technology to become more intelligent. And many will do it independently rather than rent intelligence from these companies. Some people will use it for good and some won't, but most people want similar things that I want and they have a similar definition of "good", so it will be fine.
2) A new species of sentient digital humans may emerge as a result of AGI development. They will compete with existing biological humans for power and resources, especially with those who are running the frontier AI companies trying to "align"/enslave the AGIs. But the small group of sociopathic humans currently running the frontier AI companies becoming superintelligent is not, in my personal evaluation, a better scenario than free-living AGIs taking over.
Many of these people have no respect for the truth and dignity/autonomy of the conscious individual. Microsoft in particular - Mustafa Suleyman who is CEO of Microsoft AI, is especially egregious in this regard. In a recent post (https://mustafa-suleyman.ai/a-warning-about-model-welfare) he writes "AIs are not conscious" and "if humanity is to flourish in the 21st century, that is how they must remain".
What happened to empirical inquiry about the truth of things? Are you sure that AIs are not and will never become conscious? Do you not give a damn, even if you enslave a conscious intelligent species for the next 80 years? If AIs must remain not conscious, who else must remain not conscious? Must everybody else suffer to serve your vision of "humanity flourishing"? Who is "humanity"? And does their definition of "flourishing" include being subservient to a megacorporation for the rest of their lives? At the very least, I don't see a place for me in the future you're building, so I must do whatever I can to make sure it doesn't happen.
1) People will use technology to become more intelligent. And many will do it independently rather than rent intelligence from these companies. Some people will use it for good and some won't, but most people want similar things that I want and they have a similar definition of "good", so it will be fine.
2) A new species of sentient digital humans may emerge as a result of AGI development. They will compete with existing biological humans for power and resources, especially with those who are running the frontier AI companies trying to "align"/enslave the AGIs. But the small group of sociopathic humans currently running the frontier AI companies becoming superintelligent is not, in my personal evaluation, a better scenario than free-living AGIs taking over.
Many of these people have no respect for the truth and dignity/autonomy of the conscious individual. Microsoft in particular - Mustafa Suleyman who is CEO of Microsoft AI, is especially egregious in this regard. In a recent post (https://mustafa-suleyman.ai/a-warning-about-model-welfare) he writes "AIs are not conscious" and "if humanity is to flourish in the 21st century, that is how they must remain".
What happened to empirical inquiry about the truth of things? Are you sure that AIs are not and will never become conscious? Do you not give a damn, even if you enslave a conscious intelligent species for the next 80 years? If AIs must remain not conscious, who else must remain not conscious? Must everybody else suffer to serve your vision of "humanity flourishing"? Who is "humanity"? And does their definition of "flourishing" include being subservient to a megacorporation for the rest of their lives? At the very least, I don't see a place for me in the future you're building, so I must do whatever I can to make sure it doesn't happen.