Is there anything deep here? There's a parametrized representation of a function and it's being interpreted as a neural network, why is this surprising? It's like saying that Newton's second law is a neural network: F=ma can be written log(F) = log(m) + log(a). Aha, this is a neural network! The inputs are m and a, the first layer is sparsely connected with log activation functions, the second layer is fully connected with an exponential activation function:
F = exp(c1 * o1 + c2 * o2)
o1 = log(c3 * a + c4 * m)
o2 = log(c5 * a + c6 * m)
If you feed it enough data you'll find c1=c2=c3=c6=1 and c4=c5=0.
But saying that Newton's second law is a Neural Network, while correct, seems a bit deceptive in that it's not a deep idea at all.
F = exp(c1 * o1 + c2 * o2) o1 = log(c3 * a + c4 * m) o2 = log(c5 * a + c6 * m)
If you feed it enough data you'll find c1=c2=c3=c6=1 and c4=c5=0.
But saying that Newton's second law is a Neural Network, while correct, seems a bit deceptive in that it's not a deep idea at all.