In this video interview, conversion expert Lance Loveday explains the pros and cons of simple A/B split-testing vs. the much more complex multivariate testing:
We’ve found, says, Loveday, that A/B testing tends to deliver the largest gain. Of course, he adds, that this could be A/B/C/D testing – 4 variations of a single factor -as opposed to testing a large number of different factors at one time.
The advantage of A/B testing is that it tends to cause you to rethink the entire user experience as you look for elements to test — layout, visual hierarchy, messaging, etc. Multivariate testing, on the other hand, often leads to testing smaller incremental variations that don’t make as much difference. A/B testing forces you to be more creative about what you’re testing.
Loveday explains that his firm’s process is to start with some A/B testing to find an effective layout for a client. Once that’s established, they’ll do multivariate testing to get some additional incremental gains.