A/B test example

Note - until further notice, the following patch is required: #1949122: Some contribs need core file.inc/theme.inc included in boot phase.

A/B test by definition is a test with two versions of an element (A and B) and a metric that defines success. A version is normally your existing design (“control”); and B version is the “challenger” with one copy or design element changed. A classical example is using an A/B test to compare two versions of a landing page, to find out which alternative leads to more registrations.

A/B test example using Multivariate

Hypothesis - a bigger registration link on the home page will increase the number of people who sign up for the site.
This assumes that you have set up a Drupal site that has a link to registration page (/user/register) on the home page. For this example the link is provided as part of the user login block provided by user.module.

1. Enable Multivariate, Internal CSS mutation, Path success condition modules.
2. Go to multivariate/add and add test and description for the test.
3. Under run conditions add the following conditions:
   • front page
   • User:role and check "anonymous user"

   Make sure that the run conditions is set to "All criteria must pass"
   

4. Under Variants make sure you have one set and two variants. Give the first variant name "Control" and use "Challenger" for variant 2.
   Add internal css mutation to the variants. Leave the field for css overrides blank for variant 1 and insert the following for variant 2:
   #user-login-form ul li.first a {font-weight: bolder};

   

5. Set success condition to path and insert path as user/register
   

Anonymous users hitting the home page will now see one of these two variations:
The default homepage

The modified challenger homepage