A/B testing and Google Analytics experiments

A/B testing has been a growing trend among web content editors and marketers in the last few years.

by Edwin Guilbert, Consulting Manager Spain at Magnolia España
ago 16, 2016 | 5 minutes

A/B testing has been a growing trend among web content editors and marketers in the last few years. It's not a secret that high performing webpages will lead to more page visits, lower bounce rates and more conversions, which usually means more revenue. However, we need to find out what we need to achieve with a specific page or content and try to measure it before performing any comparison tests.

We can measure how the web page is performing through metrics. Some examples are:

Call to Actions (clicking on a button, playing a video)

Product purchases

E-mail signups

From these metrics you could calculate conversion rates, which basically state how many visitors have turned into customers in a given time frame. Examples of conversions could be visitors buying products on an e-commerce website or readers clicking on advertisements in a blog website.

You may be wondering what A/B testing has to do with any of this? Well, the answer is simple: In A/B testing you could improve your conversions by comparing different versions of the same page. The idea is to change key elements that might improve your conversion rate. You usually have a original page (a so called control page) and a variation of it. The users will get any of these versions randomly and after a period of time you can compare which version did better according to a goal (or conversion).

But we're not here just to explaining how A/B testing works. We actually want to do some A/B testing in a real web page with a concrete CMS and analytics software. So we will use Magnolia CMS for managing web pages and Google Analytics Experiments for recording/analyzing statistics.

In Magnolia you can create variations of a page and edit them independently (this is only available on Enterprise editions.)

We are going to work with the demo website travels, specifically with the "about" page which contains a video. The goal here would be get this video played more times, so the metric we are going to use will be the play event on this element of the dom.

The original page looks like this:

Magnolia travels page

And the variation will have a wider video, removing the left side links which might distract the visitor and prevent her from playing the video (our goal in this case.)

Magnolia travels page

To create a variation of a page, you just have to select "Add page variant" in the pages app of Magnolia.

Magnolias´s author CMS

Open the variant created and update the video component:


After publishing the variant, you need to connect these pages to Google Analytics using a Javascript snippet for tracking page views. You can embed this snippet with Marketing Tags in Magnolia (don't forget to include it in all your pages):

Long code

After publishing the snippet (or marketing tag in Magnolia) you need to send an event for every time a user plays the video included in the pages we want to test, so we need to create another marketing tag for that:

Short code

After publishing the snippet, you need to actually record the number of play video events as conversions or "goals" in Google Analytics:

Goal setup screenshot
Goal details screenshot

After the goal is configured, you might want to test it in the real time tab of Google Analytics, so every time you play a video in your page, an event gets registered as a conversion:

Google Analytics screenshot

At this point you finally have everything prepared to start your A/B testing on the page variants measuring video played metrics as a goal. You can automatically create experiments (as Google calls them) in Magnolia using the A/B testing module:

Magnolia CMS Author Screenshot

As shown in the screenshot below, you can define the name of the experiment, the page, and variants involved, the duration and the goal for the test. The goal is dynamically retrieved from the Google Analytics account configured:

Magnolia CMS Author Screenshot

The module will create a Javascript snippet as a marketing tag, which you need to publish, so Google can forward users to one variant or the other:

Magnolia CMS Author Screenshot

After publishing the snippet you can already start the experiment in Google Analytics through the A/B testing module:

Magnolia CMS Author Screenshot

And that's all folks, you can always check your statistics in Google Analytics behaviour tab:

Google Analytics Screenshot

I hope you enjoyed my first post and find this useful. I will write a second part of this post explaining how to do A/B testing manually without using the A/B testing and Personalisation modules (both need the Enterprise version of Magnolia).

Edwin Guilbert

About the Author

" His added role covers the following areas (not limited to): Coordinating all the tasks related to trainings, audits, workshops and pre-sales work, deciding who is the appropriate person on the team to take on each task. Coordinating with Key Account Management, Partner Management and Sales in the Spanish Office, to make sure we are using our capacity in the most efficient way. Coordinating with the Spanish Marketing Team on all the activities related to foster the developers community in the Spanish office territories. Coordinating with Global PD regarding new features, add-ons, technical customer feedback and needs, etc. Coordinating with Global Support in terms of servicing particular customer needs. Providing capacity to HQ when needed, both on trainings and consulting services, in coordination with the local activities. Executing some of the tasks as Senior Software Architect and Trainer."