Comment by bitL

I think the issue with Deep Learning is that it is like a hyperdimensional optimization heuristics sequence surpassing what human mind can comprehend and pushing limits of computing (depth of 100 layers max at the moment). Given how difficult are far more trivial optimization techniques and proving bounds for them, it seems the times where we could just define a new approach and prove some nice properties are behind us :-(