Comment by kristopolous

I'm still very much learning this stuff, but I wonder if that's related to the vanishing gradient problem, which seems to be a fundamental aspect of these types of approaches. (Please don't assume that's correct)

https://en.wikipedia.org/wiki/Vanishing_gradient_problem