> once we have trained a large overparameterized neural network we can often compress it to a smaller one with very little loss. So why can't we learn the smaller one directly?
I feel something similar goes on in us humans. Interesting to think about.
I feel something similar goes on in us humans. Interesting to think about.