Lots of folks/companies are going to lose long-term as a result of bad A/B testing driven decision making. I suspect the decline of the Facebook feed might be attributed to this as well (maybe even the recent polls of Trump's twitter practices[1]). Generally these sorts of things don't really consider long term fatigue and other important factors, and take short term micro-benchmark wins as success. On the flip side, some attribute not A/B testing to the failure of the SnapChat redesign[2].
Is there anyway to do this properly? I can imagine the problem will get worse and worse as we reduce human evaluation of A/B test results and automatically make real-time decisions through software and ML.
Is there anyway to do this properly? I can imagine the problem will get worse and worse as we reduce human evaluation of A/B test results and automatically make real-time decisions through software and ML.
[1] https://www.businessinsider.com/poll-shows-trump-appears-to-...
[2] https://www.engadget.com/2018/10/05/snap-evan-spiegel-app-re...