I recently did a deep dive on open-endedness, and my favorite example of its power is Picbreeder from 2008 [1]. It was a simple website where users could somewhat arbitrarily combine pictures created by super simple NNs. Most images were garbage, but a few resembled real objects. The best part is that attempts to replicate these by a traditional hill-climbing method would result in drastically more complicated solutions or even no solution at all.
It's a helpful analogy to understand the contrast between today's gradient descent vs open-ended exploration.
It's a helpful analogy to understand the contrast between today's gradient descent vs open-ended exploration.
[1] First half of https://www.youtube.com/watch?v=T08wc4xD3KA
More notes from my deep dive: https://x.com/jinaycodes/status/1932078206166749392