Comment by colincsl

I've only been watching from the Deep Learning sidelines -- but I believe people have steered away from pretraining over the past year or two. I think on practical datasets it doesn't seem to help.