Biggest trap of Simpson's paradox is the results can change with every level of granularity.
If you take the example of Treatment A vs Treatment B for tumors, you can get infinite layers of seemingly contradicting statemens:
- Overall, Treatment A has better average results
- But if you add tumor size, Treatment B is always better
- But if you add gender to size, Treatment B is always better
- But if you add age category to gender and size, Treatment A is always better
- etc...
It totally contradicts our instincts, and shows statistics can be profoundly misleading (intentionally or not).
If you take the example of Treatment A vs Treatment B for tumors, you can get infinite layers of seemingly contradicting statemens: - Overall, Treatment A has better average results - But if you add tumor size, Treatment B is always better - But if you add gender to size, Treatment B is always better - But if you add age category to gender and size, Treatment A is always better - etc...
It totally contradicts our instincts, and shows statistics can be profoundly misleading (intentionally or not).