> And one possibility might be that AIs could “improve themselves” to produce a single “apex intelligence” that would in a sense dominate everything else. But here we can see computational irreducibility as coming to the rescue. Because it implies that there can never be a “best at everything” computational system. It’s a core result of the emerging field of metabiology: that whatever “achievement” you specify, there’ll always be a computational system somewhere out there in the computational universe that will exceed it.
That tigers are stronger than wolves doesn't prevent wolves from killing you.
Wolfram makes that class of error repeatedly. Another example: what prevents a system of ethics from being both Godel-incomplete/inconsistent and at the same time encompassing everything in the realm of human experience?
> For the first time in history, it’s become realistic to truly automate intellectual tasks. The leverage this provides is completely unprecedented.
The essay also mentions how humans are still the best at manual labor, and how government can work with humans directing the AI as it runs things every more efficiently. It doesn't touch on which humans wield that leverage, naively assuming it'll be all of us in a cozy democracy.
The author is too used to thinking from first principles and hasn't studied enough human history to understand how leverage gets abused.
That tigers are stronger than wolves doesn't prevent wolves from killing you.
Wolfram makes that class of error repeatedly. Another example: what prevents a system of ethics from being both Godel-incomplete/inconsistent and at the same time encompassing everything in the realm of human experience?
> For the first time in history, it’s become realistic to truly automate intellectual tasks. The leverage this provides is completely unprecedented.
The essay also mentions how humans are still the best at manual labor, and how government can work with humans directing the AI as it runs things every more efficiently. It doesn't touch on which humans wield that leverage, naively assuming it'll be all of us in a cozy democracy.
The author is too used to thinking from first principles and hasn't studied enough human history to understand how leverage gets abused.