Comment

Comment by Moshe_Silnorin

What people are worried about is artificial systems that are of higher capability than all or the vast majority of humans. If the principal agent problem is solved by those that create them and they can be cheaply reproduced, then the value of human wages goes below subsistence. This likely isn't a big problem as economic growth rates would be so unimaginably high in this scenario (with doubling times on the order of months) that even significantly less wealth distribution than what occurs today could easily cover a basic income. This is why I'm not nearly as worried about technological unemployment. And as most people have at least some capital, even without redistribution many people would be fine.

If we can't solve the principle agent problem, we will have introduced self-replicating entities with much higher intelligence into our environment. As most possible utility functions require resources to pursue, we will be competing for resources with more intelligent entities. A competition we will lose.

So solving the principal agent problem is a big issue. Comparing human-level AI to markets is like Megafauna saying "We've been competing with other mammals for millions of years, man is only a difference in degree rather than kind. We'll be fine."

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton