Described in this way, it feels like a reasonable argument that humanity is like a paperclip-optimizing AI that just maybe just chooses random things to go deep on optimizing.
We created a problem for ourselves that was fun -- games -- but once we'd beaten "games", the reward function kept running and now we're adversarially inventing new ways to "optimally" beat harder and harder versions of "games", ie new objectives, new limitations, new allowable orderings of elements within the games, etc.
We created a problem for ourselves that was fun -- games -- but once we'd beaten "games", the reward function kept running and now we're adversarially inventing new ways to "optimally" beat harder and harder versions of "games", ie new objectives, new limitations, new allowable orderings of elements within the games, etc.
I wonder what 50 years of this will yield!