Human-level intelligence is highly variable. If we’re talking about something that can intelligently and independently make discoveries in math science and engineering, then the comparative advantage can’t stay long because you’d just pose the problem of “make yourself but faster” to it.
It doesn’t even need discoveries. Imagine an AI that could manually transcode your high level code into absolutely optimal assembly, simplifying your design, removing unnecessary code, optimizing the code in response to observed behavior, etc. I would guess we have several orders of magnitude of power efficiency loss just from building a system that has understandable and flexible layers of abstraction. A sufficiently powerful AI could work to automatically remove those abstractions and even operate at a higher level of abstraction.
The real question is whether we’re at all on the right track. ¯\_(ツ)_/¯
It doesn’t even need discoveries. Imagine an AI that could manually transcode your high level code into absolutely optimal assembly, simplifying your design, removing unnecessary code, optimizing the code in response to observed behavior, etc. I would guess we have several orders of magnitude of power efficiency loss just from building a system that has understandable and flexible layers of abstraction. A sufficiently powerful AI could work to automatically remove those abstractions and even operate at a higher level of abstraction.
The real question is whether we’re at all on the right track. ¯\_(ツ)_/¯