Comment by achompas

I actually wrote a blog post on regularization a few months ago. I think it covers some of the issues, like overfitting, glossed over in this post:

http://acompa.net/on-regularization.html