A lot of the time you are running experiments and seeing how the data changes, not looking at the data in a vacuum.
Many of the example critiques here don't apply so much when looking at the changes in data:
- if I got 50% more hits on my site this week vs last week, that's meaningful despite 36% of people blocking ads
- if my open rate doubled when I changed my email subject, also meaningful
The other examples are hard to pick holes in as they simply say "Z is a lie", but I can be looking at multiple data sources to decide how much of a lie Z is.
Many of the example critiques here don't apply so much when looking at the changes in data:
- if I got 50% more hits on my site this week vs last week, that's meaningful despite 36% of people blocking ads
- if my open rate doubled when I changed my email subject, also meaningful
The other examples are hard to pick holes in as they simply say "Z is a lie", but I can be looking at multiple data sources to decide how much of a lie Z is.