Comment by smartmic

In my experience, overfitting also helps in case of small data problems. After all, there is a reason that you do only have few training data samples and the chance that any new prediction sample is close to the existing training ones is higher. In other words, for small data its more about specialization than generalization. Of course this has to be evaluated case specific.