I've only recently started reading about deep neural networks, and the thing that strikes me the most about the literature is the lack of mathematics.
Open a NIPS paper from 2010 or so, and you'll see extremely dense mathematics: nonparametrics, variational approximation, sampling theory, riemannian geometry. But from my (admittedly small) sampling of the convnet / RNN literature there really doesn't seem to be much maths there. The typical paper seems to run along the lines of "We tried this, and it worked".
I'm not sure whether there's anything to learn from this observation, but I think it's striking all the same.
Open a NIPS paper from 2010 or so, and you'll see extremely dense mathematics: nonparametrics, variational approximation, sampling theory, riemannian geometry. But from my (admittedly small) sampling of the convnet / RNN literature there really doesn't seem to be much maths there. The typical paper seems to run along the lines of "We tried this, and it worked".
I'm not sure whether there's anything to learn from this observation, but I think it's striking all the same.