Ask HN: What are the cons of experimentation driven products?

Experimentation (like A/B) testing is a big part of product development in the industry. I see plenty of articles about the goodness of experimentation and was wondering what are the cons of experimentation driven development to keep in mind.