While we're at it, doesn't the "universality theorem" (as the article calls it) basically follow immediately from the fact that the set of all continuous functions comprises a vector space?
If the continuous function is additive, it's linear. If it's nonlinear, you can differentiate it to obtain a linear approximation. A neural network computes linear transformations, so unless I'm missing something I'm a little surprised there's a substantive theorem for this. Is it not a corollary on the fact that we can construct a vector space of all continuous functions?
If the continuous function is additive, it's linear. If it's nonlinear, you can differentiate it to obtain a linear approximation. A neural network computes linear transformations, so unless I'm missing something I'm a little surprised there's a substantive theorem for this. Is it not a corollary on the fact that we can construct a vector space of all continuous functions?