Comment by jrkatz

For very long term effects (e.g., the impact of painting every surface of your website in ads which will certainly drive users away), hold backs aren't enough because it's too hard to consistently identify users. If ads will drive me away over the course of a year, there's a five percent holdback, and I use multiple devices and cash maybe be consistently identified for a few days or weeks at a time, that means that 95% of the time I use the website I get the bad experience, and I eventually bounce. The holdback will show no effect.