I've consulted in many situations where people make horrible business decisions based on "statistically significant." People mistake p-values for p(hypothesis|data), mistake p-values for effect size ("it's highly significant!"); moreover, people using NHST don't understand multiplicity or "topping-off" problems.
Of course one could argue that they just need to review Intro Stats, but that skips over the impenetrable conceptual nature of p-values and NHST. Given that regular people must use applied stats and given that NHST is fundamentally arcane and confusing, then the stage is set for endless drama.
Now if we were sampling manufactured products from batches and looking for a quantitative upper-bound on the failure rates, run over many batches, then we have a winner!
I've consulted in many situations where people make horrible business decisions based on "statistically significant." People mistake p-values for p(hypothesis|data), mistake p-values for effect size ("it's highly significant!"); moreover, people using NHST don't understand multiplicity or "topping-off" problems.
Of course one could argue that they just need to review Intro Stats, but that skips over the impenetrable conceptual nature of p-values and NHST. Given that regular people must use applied stats and given that NHST is fundamentally arcane and confusing, then the stage is set for endless drama.
Now if we were sampling manufactured products from batches and looking for a quantitative upper-bound on the failure rates, run over many batches, then we have a winner!