Comment by akst

In most cases the A and the B in an A/B test really should otherwise mostly identical, but one thing is different.

If almost everything is different, it’s hard to learn for next time what exactly what led to a change in which ever dependent variable your observating.