I agree with the main point of the article, but I'm somewhat disturbed by the statistical errors and misconceptions.
>Few websites actually get enough traffic for their >audiences to even out into a nice pretty bell curve. If >you get less than a million visitors a month your >audience won’t be identically distributed and, even >then, it can be unlikely.
What is the author trying to say here? Has he thought hard about what it means for "an audience" to be identically distributed?
>Likewise, the things that matter to you on your website, >like order values, are not normally distributed
Why do they need to be?
>Statistical power is simply the likelihood that the >difference you’ve detected during your experiment >actually reflects a difference in the real world.
Simply googling the term would reveal this is incorrect.
>Few websites actually get enough traffic for their >audiences to even out into a nice pretty bell curve. If >you get less than a million visitors a month your >audience won’t be identically distributed and, even >then, it can be unlikely.
What is the author trying to say here? Has he thought hard about what it means for "an audience" to be identically distributed?
>Likewise, the things that matter to you on your website, >like order values, are not normally distributed Why do they need to be?
>Statistical power is simply the likelihood that the >difference you’ve detected during your experiment >actually reflects a difference in the real world. Simply googling the term would reveal this is incorrect.