Comment by fg6hr

A genuine question: is there any math behind ML at all? For example, is there any solid theory, with proven theorems, that would tell us what happens when we add another conv layer here or use a 3x3 conv kernel instead of a 2x2 one over there, or replace that tanh with a relu? From my limited understanding, ML roughly works like this: we shuffle around the ML graph, using some intuition, off-load it to a cluster of GPUs that costs $10k/hour, feed it a dataset with 1 billion images and see what happens; but nobody can predict the behavior of training or convergence or accuracy based on the ML graph and data alone.