This seems terribly dogmatic to me. Take the example of clinical trials. Suppose you're testing a new cancer drug. You design an experiment to test the new drug, named B, versus an established chemotherapy treatment named A. You expect B's performance to be similar to A's performance in controlling the cancer, so to make sure your trial has high power, you plan to test the drugs on 2,000 patients (with each drug administered to 1,000).
Now consider the following two scenarios:
(1) After giving drug B to 100 patients, all 100 patients are dead. Do you continue the trial, giving the (apparently) deadly drug B to 900 more patients?
(2) After giving drug B to 100 patients, all 100 patients are totally cured (vs A curing 3 in 100). Do you continue the trial, withholding the (apparent) cure for cancer from 900 more patients?
In either case, since you have a strong effect, it seems to me there is logical justification to end the trial early.
Obviously the stakes are higher in clinical trials than website design, but in both cases, data acquisition has costs and intermediate results may inform changes to your experiment design.
I honestly cannot see how anyone could blankly assert that stopping a test is always wrong. There are certainly circumstances where you do want to stop early. You just have to make sure you aren't misinterpreting a statistic when you do so.
Now consider the following two scenarios:
(1) After giving drug B to 100 patients, all 100 patients are dead. Do you continue the trial, giving the (apparently) deadly drug B to 900 more patients?
(2) After giving drug B to 100 patients, all 100 patients are totally cured (vs A curing 3 in 100). Do you continue the trial, withholding the (apparent) cure for cancer from 900 more patients?
In either case, since you have a strong effect, it seems to me there is logical justification to end the trial early.
Obviously the stakes are higher in clinical trials than website design, but in both cases, data acquisition has costs and intermediate results may inform changes to your experiment design.
I honestly cannot see how anyone could blankly assert that stopping a test is always wrong. There are certainly circumstances where you do want to stop early. You just have to make sure you aren't misinterpreting a statistic when you do so.