> > Stripped of anything else, neural networks are compositions of differentiable primitives
> I’m a sucker for statements like this. It almost feels philosophical, and makes the whole subject so much more comprehensible in only a single sentence.
And I hate inaccurate statements like this. It pretends to be rigorous mathematical, but really just propagates erroneous information, and makes the whole article so much more amateur in only a single sentence.
The simple relu is continuous but not differentiable at 0, and its derivative is discontinuous at 0.
> I’m a sucker for statements like this. It almost feels philosophical, and makes the whole subject so much more comprehensible in only a single sentence.
And I hate inaccurate statements like this. It pretends to be rigorous mathematical, but really just propagates erroneous information, and makes the whole article so much more amateur in only a single sentence.
The simple relu is continuous but not differentiable at 0, and its derivative is discontinuous at 0.