Just reading it now -- seems like a very thorough takedown of the whole thing, in fairly non-technical language.
It makes the very good point that this paper makes lots of statistical tests, and then bases big claims on the small minority that showed a significant effect. This is in no way restricted to psychology, drugs companies for example do this all the time. It's cheating, whether you realize it or not.
Statistical significance only tells you that a result is unlikely to be a fluke -- not that it definitely isn't a fluke -- but the more tests you do, the sooner you'll see a fluke on average.
It makes the very good point that this paper makes lots of statistical tests, and then bases big claims on the small minority that showed a significant effect. This is in no way restricted to psychology, drugs companies for example do this all the time. It's cheating, whether you realize it or not.
Statistical significance only tells you that a result is unlikely to be a fluke -- not that it definitely isn't a fluke -- but the more tests you do, the sooner you'll see a fluke on average.