Comment by the_grue

I think a important advance is being able to build neural networks in a modular fashion by constructing a toolkit of components and techniques one can reasonably expect, when combined, to produce certain result. As the field advances, this toolbox will become richer and better defined and delineated, forming basic building blocks. Just like functions, loops and virtual methods came out of the chaos of assembly.