Comment by L2R

There are approaches to ensure parameters remain stable despite the depth (selu, for example).