“Using modern experiment frameworks, all 3 of ideas can be safely tested at once, using parallel A/B tests (see chart).”
Nooo! First, if one actually works, you’ve massively increased the “noise” for the other experiments, so your significance calculation is now off. Second, xkcd 882.
Nooo! First, if one actually works, you’ve massively increased the “noise” for the other experiments, so your significance calculation is now off. Second, xkcd 882.