> What I deeply believe is: We're never going to invent superintelligence, not because its impossible for computers to achieve, but because we don't even know what intelligence is.
Speak for yourself, not all of humanity. There are plenty of rigorous, mostly equivalent definitions for intelligence: The ability to find short programs that explain phenomena (compression). The capability to figure out how to do things (RL). Maximizing discounted future entropy (freedom). I hate how stupid people propagate this lie that we don't know what intelligence is, just because they lack it. It's quite convenient, because how can they be shown to lack intelligence when the word isn't even defined!
Speak for yourself, not all of humanity. There are plenty of rigorous, mostly equivalent definitions for intelligence: The ability to find short programs that explain phenomena (compression). The capability to figure out how to do things (RL). Maximizing discounted future entropy (freedom). I hate how stupid people propagate this lie that we don't know what intelligence is, just because they lack it. It's quite convenient, because how can they be shown to lack intelligence when the word isn't even defined!