I saw a great talk about this @ SIAM data mining by Dave Madigan, for http://omop.org/
Basically, they took a TON of demographic research in the health sciences, explored all possible hyperparameter tunings, and found that they could get p-values of 0.05 in either direction for most of the papers depending on choice of data sources and many other types of hyperparameters.
Basically, they took a TON of demographic research in the health sciences, explored all possible hyperparameter tunings, and found that they could get p-values of 0.05 in either direction for most of the papers depending on choice of data sources and many other types of hyperparameters.