Comment by sfvisser

Humans maybe wrote the code, but not the network of weights on top. And that’s where the magic happens.

Even if we’d understand precisely how every neuron in our brains work at a molecular level there is no reason to believe we’d understand how we think.

We can’t simply reduce one layer into another and expect understanding.