The experiments we conducted emphasize that the effective capacity of several successful neural network architectures is large enough to shatter the training data. Consequently, these models are in principle rich enough to memorize the training data.
So they're fitting elephants[1].
I've been trying to use DeepSpeech[2] lately for a project, would be interesting to see the results for that.
I guess it could also be a decent test for your model? Retrain it with random labels and if it succeeds the model is just memorizing, so either reduce model complexity or add more training data?
So they're fitting elephants[1].
I've been trying to use DeepSpeech[2] lately for a project, would be interesting to see the results for that.
I guess it could also be a decent test for your model? Retrain it with random labels and if it succeeds the model is just memorizing, so either reduce model complexity or add more training data?
[1]: https://www.johndcook.com/blog/2011/06/21/how-to-fit-an-elep...
[2]: https://github.com/mozilla/DeepSpeech