Can't really agree much on this article and I have the sense it is somehow written out of frustration and lack of experience.
The '6 hidden costs' can all be addressed, some are not even hidden like time, running tests take time and resources like anything else. The other factors like speed/performance and confusion can all be solved very easily.
Some of my take-aways for testing
- Don't focus on small changes like buttons, copy etc.. these are mostly useless yes
- Make radical changes, e.g. on stickermule can try to have the whole product to make a sticker to checkout on 1 page or start with configuration first and then select sizes etc..
- To filter out false positives, compare same sample of data (e.g. organic traffic is good) for the same period range before a winning test went live and after
Personally I have seen some great results with the above tips in mind, from B2B making visitors perform small actions first or showing a demo vs. presenting them with a form immediately (4x more leads) to e-commerce sites adding a direct checkout vs. normal checkout path (lower avg. order but much higher conversion rate to largely make up for it) etc..
You could argue these are part of product (which a conversion department should be part of, not marketing) but without doing the testing you can only guess about the outcome. And sure many tests are fixing usability (e.g. seen 250% increase in registrations by fixing telephone number prefix and date of birth field formatting) and UI issues but that's part of it.
The '6 hidden costs' can all be addressed, some are not even hidden like time, running tests take time and resources like anything else. The other factors like speed/performance and confusion can all be solved very easily.
Some of my take-aways for testing
- Don't focus on small changes like buttons, copy etc.. these are mostly useless yes
- Make radical changes, e.g. on stickermule can try to have the whole product to make a sticker to checkout on 1 page or start with configuration first and then select sizes etc..
- To filter out false positives, compare same sample of data (e.g. organic traffic is good) for the same period range before a winning test went live and after
Personally I have seen some great results with the above tips in mind, from B2B making visitors perform small actions first or showing a demo vs. presenting them with a form immediately (4x more leads) to e-commerce sites adding a direct checkout vs. normal checkout path (lower avg. order but much higher conversion rate to largely make up for it) etc..
You could argue these are part of product (which a conversion department should be part of, not marketing) but without doing the testing you can only guess about the outcome. And sure many tests are fixing usability (e.g. seen 250% increase in registrations by fixing telephone number prefix and date of birth field formatting) and UI issues but that's part of it.