Many concerns in this paper, especially about deep learning being data hungry, limited capacity for transfer, and integrating prior knowledge are addressed by recent papers in meta-learning and learning-to-learn [1,2,3 and many many others] both in the supervised and reinforcement learning contexts.
In the case of meta-reinforcement learning there has been recent work [5] which seems to indicate that this mechanism is very similar to how learning works in the brain.
In fact, my group recently published a paper [4] about learning-to-learn in spiking networks (making it biologically more realistic) and showing that the network learns priors on families of tasks for supervised learning, and learns useful exploration strategies automatically when doing meta-reinforcement learning.
While I don't claim this is the right path to AGI, it's a very promising and new direction in deep learning research which this paper seems to ignore.
In the case of meta-reinforcement learning there has been recent work [5] which seems to indicate that this mechanism is very similar to how learning works in the brain.
In fact, my group recently published a paper [4] about learning-to-learn in spiking networks (making it biologically more realistic) and showing that the network learns priors on families of tasks for supervised learning, and learns useful exploration strategies automatically when doing meta-reinforcement learning.
While I don't claim this is the right path to AGI, it's a very promising and new direction in deep learning research which this paper seems to ignore.
[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.5.32...
[2] https://arxiv.org/abs/1611.05763
[3] https://arxiv.org/abs/1703.03400
[4] https://arxiv.org/abs/1803.09574
[5] https://www.nature.com/articles/s41593-018-0147-8 (Preprint: https://www.biorxiv.org/content/early/2018/04/06/295964)