Comment by tedsanders

Running experiments for preset lengths is a mistake. If an effect is strong, it will show up earlier. If this is the case, you want to be able to switch earlier. If you are running a drug trial of drug A vs a control, and drug A kills 100% of the first 100 patients who take it while the control kills 0, you end the trial immediately. You don’t continue to give it to 900 patients just because you pre-registered to treat 1,000 patients, thinking that the effect would be small. This is one reason I think Bayesian approaches are better than frequentist approaches for A/B testing.