We actually did test. We didn't test in the exact same conditions or environment (i.e with no internet access), as we didn't find out about the no-internet restriction until the last minute. That is covered in this paragraph:
> While this bug was a costly mistake, we learned from it. Whenever we would deploy code-last minute, we'd try to test it more rigorously. If we were running a study without internet access, we'd make sure to test in the same environment. We hadn't accounted for the environment change, partially due to the short notice for the locked-down machine, but also just because we didn't test with the exact same restrictions.
> While this bug was a costly mistake, we learned from it. Whenever we would deploy code-last minute, we'd try to test it more rigorously. If we were running a study without internet access, we'd make sure to test in the same environment. We hadn't accounted for the environment change, partially due to the short notice for the locked-down machine, but also just because we didn't test with the exact same restrictions.