We do understand how it works most of the time, but can't predict if certain changes will be beneficial or detrimental. After the fact it's pretty clear what e.g. CNNs do (find manifold transforms which, coupled with nonlinearities minimize error in the layer-wise output distributions when backpropagating loss of the minibatches of the training set). But you can't reliably say "if I add e.g. a bottleneck branch over here my accuracy will go up/down X%".
Fundamentally we have to contend with the fact that human ability to understand complex systems is fairly limited, and at some point it will become an impediment to further progress. Arguably in a number of fields we're past that point already.
Fundamentally we have to contend with the fact that human ability to understand complex systems is fairly limited, and at some point it will become an impediment to further progress. Arguably in a number of fields we're past that point already.