To avoid big A/B testing mistakes, perhaps you shouldn't use phrases like "To counter that noise it is important to first “prove the Null hypothesis.” To prove the Null hypothesis you..."
One of the most important underlying statistical principles in inference tests is that the null hypothesis can never be proven. Any data you collect can only reject the null hypothesis or fail to reject it.
One of the most important underlying statistical principles in inference tests is that the null hypothesis can never be proven. Any data you collect can only reject the null hypothesis or fail to reject it.