A/B-Testing

e-Spirit AG

2019-01-17
Table of Contents

1. Introduction

FirstSpirit can be used to create complex websites. These websites are usually created based on a range of subjective decisions. Subjective decisions will also govern design.

The design of a website aims to stimulate interaction on the part of visitors to the site by the specific placing of calls-to-action. These calls-to-action might include a request to subscribe to a newsletter, to download a document, or to purchase a product, for example. This approach is designed to maximize the conversion rate. The conversion rate provides information about how many visitors respond to a call-to-action and in doing so become potential customers.

To achieve this aim, developers must constantly assess how and where to place calls-to-action on a website in order to have the most resonance with visitors. This can seldom be decided on the basis of pre-existing information; developers must work it out for themselves. The more information developers have about visitors and the better they know them through this information, the more accurately they will be able to predict their behavior.

This is exactly where the A/B-Testing module comes in. The module can be used to show a variety of visitors several different variants of one website in the context of an experiment. The different variants of the website are distributed uniformly or according to a predefined rate. They appear again when the page is called up again subsequently. Therefore, the individual visitors are unaware that they are taking part in the experiment.

When the module is used in conjunction with an analytics tool, important statistical information can be gathered about visitors' behavior. This approach is also referred to as data-driven marketing. It provides developers with feedback about the success of each variant of the website. They can see which change to the design achieves the best results in terms of maximizing the conversion rate. At the end of an experiment, developers can apply this change directly by selecting the winning variant as the new original site.

When making changes in the future, developers will no longer have to rely on their gut instincts. They can start other experiments or use information already gathered in order to continuously optimize their websites.

The A/B-Testing module has the following technical requirements:

  • FirstSpirit (Isolated- oder Legacy-Mode) starting version 2018-11
  • Java 8 or higher

2. Range of functions

With the A/B-Testing module, editors can:

  • Create and carry out experiments
  • Add any number of variants to experiments
  • Define a distribution rate for each individual variant
  • Specify project-specific participation rate

The corresponding functions are made available in both ContentCreator and the SiteArchitect preview when the module is installed and configured.

The A/B-Testing module functions cannot be used in conjunction with the external FirstSpirit preview.

Familiar FirstSpirit tools are used to set up and maintain an experiment. Therefore, editors are not expected to have any knowledge in addition to being familiar with FirstSpirit.

An experiment is carried out with a release workflow. The workflow must include all of the pages and page references involved in an experiment. A workflow of this type is supplied with the A/B-Testing module. Based on the BasicWorkflows, it meets the specified requirements. If a project-specific workflow is to be used instead, it must be adapted accordingly with the API provided.

Tracking during the runtime of an experiment is essential in order to analyze the success of each individual variant. This is not part of the module and must be carried out on a project-by-project basis. An analytics tool (freely chosen by the user) is required for this purpose. In this regard, the A/B-Testing module has a completely open architecture and is not limited to specific services.

A tracking format template plug-in regulates the connection between the analytics tool and the FirstSpirit server. A plug-in of this type is supplied with the module as standard. It is set up for Google Analytics and takes little time and effort to configure for immediate use. Developers who wish to use a different analytics tool should implement a corresponding tracking plug-in.

When a page which is part of an experiment is called up, the A/B-Testing module creates cookies. Therefore, we recommend drawing the attention of users to the use of cookies (in accordance with the EU's ePrivacy Directive and - specifically in Germany - Section 15 Paragraph 3 of the Teleservices Act), if you do not already do this.

3. Quick start guide

The quick start guide outlines the possible statuses of an experiment. It is designed for use by editors and provides them with an initial overview. Therefore, the content of this chapter has been kept brief deliberately; only the basic functions of the A/B-Testing module are described. For a detailed description of the module and additional information, see chapters Life cycle of an experiment and Use in FirstSpirit.

During its runtime, each experiment completes a specific cycle. A simplified map of this cycle is shown below (see figure Simple life cycle of an experiment).

Simple life cycle of an experiment
Figure 1. Simple life cycle of an experiment


The cycle begins with the creation and start of an experiment. This triggers a workflow which must release all of the pages and page references involved in the experiment.

On successful completion of the start, the experiment changes status to running. The cycle stops when an experiment is finished and a new original page is selected. A new experiment can potentially be created for this page.

3.1. Creating an experiment

An experiment must be created before it can be carried out. Experiments can be created in various ways in the SiteArchitect and the ContentCreator.

ContentCreator
To create an experiment for the page displayed, select ActionsCreate experiment from the menu in ContentCreator. If the experiment is to be created for a different page, you must call it up first.
SiteArchitect
To create an experiment in SiteArchitect, click the Create experiment button (see figure Create experiment button in the SiteArchitect). This button can only be used on page references. Therefore, in the SiteStore, you must first select the required page reference for which an experiment is to be carried out.
Create experiment button in the SiteArchitect
Figure 2. Create experiment button in the SiteArchitect


If the selected page reference is already involved in an experiment or if it is not permissible for creating an experiment, the Create experiment menu item or button will be hidden.

If the menu item or button is visible but deactivated, a configuration error has occurred or the permissions for carrying out an experiment are lacking. In this case you must contact the administrator.

The A/B-Testing module functions cannot be used in conjunction with the external FirstSpirit preview.

In both clients, the A/B-Testing bar appears when the experiment is created. It contains two tabs: one for the original page and another for the variant created automatically. The variant is selected for immediate editing (see figure A/B-Testing bar in ContentCreator and in the SiteArchitect preview).

A/B-Testing bar in ContentCreator and in the SiteArchitect preview
Figure 3. A/B-Testing bar in ContentCreator and in the SiteArchitect preview


3.2. Editing an experiment

Any number of new variants can be added to or existing variants can be deleted from a current experiment. Moreover, a distribution rate can be defined for each variant, along with a participation rate for the entire experiment.

Add variant
New variants can be added by selecting the Add variant button (displayed as a plus sign) or the duplicate option from the drop-down menu of each tab.
Delete variant
Existing variants can be removed from the experiment by selecting Delete variant from the drop-down menu of the corresponding tab. A confirmation prompt will be displayed if you attempt to delete the original page.

Only variants that are not currently in focus can be deleted in SiteArchitect.

Defining the participation rate and distribution rates
The Edit settings button (displayed as three intermeshing cogwheels) opens a dialog in which the participation rate and the distribution rates for the variants can be defined (see figure Edit settings). The settings dialog is described in detail in chapter Configuring an experiment.
Edit settings
Figure 4. Edit settings


3.3. Starting an experiment

Once an experiment has been created and edited, it can be started with the Start button. If the experiment starts successfully, the button turns green and its label changes to Running.

Under certain circumstances, it is possible to create and edit an experiment without then being able to start it afterwards. A message dialog box appears instead and the experiment remains in its initial status.

If the dialog box indicates that the workflow cannot be determined, the selection made when the A/B-Testing module was configured is incorrect. When an experiment is created, the information is simply checked to ensure that it exists, not for potential errors. This check is only run when an attempt is made to start an experiment.

In such cases, please contact your administrator.

3.4. Finishing an experiment

A current experiment can be stopped with the Finish experiment button (displayed as a flag). The selected variant is set as the new original page and the A/B-Testing bar disappears from view.

If an element of the experiment is still in the workflow, the experiment cannot be finished. The editor is notified accordingly and the experiment is retained.

4. Installation & configuration

Various components must be installed and configured in order to use the functions supported by the A/B-Testing module. The steps to be completed are described in the following sub-chapters.

4.1. Module installation

Use the abtesting-<version number>.fsm file supplied to add the module on the FirstSpirit Server. To install the module, open the ServerManager and select Server propertiesModules.

Module management in the server properties
Figure 5. Module management in the server properties


The main panel contains a list of modules installed on the FirstSpirit Server. After clicking Install, select the abtesting-<version number>.fsm file supplied with the module and click Open to confirm your selection. After successful installation, an A/B-Testing folder is added to the list and must be given All permissions (see figure Module management in the server properties).

After any module installation or update, the FirstSpirit Server needs to be restarted.

4.2. Configuring the project component

A number of templates and project-specific settings are necessary in order to use the A/B-Testing module. Templates can be imported and the remaining configuration settings can be made via the project component, which must be added to the project you are using. To add the project component, open the ServerManager and select Project propertiesProject components.

Project components in the project properties
Figure 6. Project components in the project properties


A list of all existing project components is displayed in the main panel. After clicking Add, select the A/B-Testing ProjectApp and click OK to confirm your selection. The project component is then added to the list in the main panel and will need to be configured (see figure Project components in the project properties). To configure the project component, select the entry in the list and click Configure to open the associated dialog (see figure Configuration dialog for the project component).

Configuration dialog for the project component
Figure 7. Configuration dialog for the project component


Workflow to start the experiment

First, specify a workflow to start an experiment in a combo box in the configuration dialog. This must be a release workflow that is to be created on a project-specific basis and releases all pages and page references involved in an experiment.

The combo box label is displayed in red until a workflow is selected. If a workflow is not specified and/or if any other configuration errors occur, it will not be possible to create an experiment. The corresponding button or menu item will be visible in both clients but not activated.

Only a null check is performed when creating an experiment. Potentially erroneous information is not relevant to this check and is not taken into account. If the workflow selected in the combo box is deleted within the project, it will still be possible to create an experiment.

The correctness of the data is not checked until an attempt is made to start, continue, or update an experiment. In such instances, a message dialog box is displayed for the editor and the experiment is paused in its current status.

When the experiment is started via the A/B-Testing bar, the workflow is only executed on the dispatcher page. All variants of the experiment must also be included in the workflow automatically (see also chapter API). This is absolutely essential in particular if the workflow contains a deployment.

Name of the prioritized transition

A workflow often contains several outgoing transitions from a single status. In such cases, the user must decide which of these transitions is to be selected. The workflow pauses here until it is continued in the current status. Accordingly, the reference name of a transition to be prioritized can be specified for the workflow selected in the project component. This transition is executed by the workflow and the other transitions are ignored.

If the transition to be prioritized cannot be identified or has not been defined, the transition with the reference name ab_testing_prioritized is executed. This is a default value. If this transition does not exist, the editor will see a corresponding message.

If the selected workflow contains just one outgoing transition for each status, a decision does not have to be made. This transition is always executed and there is no need to specify a prioritized transition in the configuration dialog.

Selecting permitted page templates

The configuration dialog also displays a list of all page templates that are available to editors. This means that technical or similar templates that are hidden in the selection lists of the two clients are not displayed here. Select the permissible templates for carrying out an experiment from the list in the configuration dialog. The button or menu item for creating an experiment will be hidden in both clients for page references which have an underlying page based on a template that has not been selected. Use the key combination CTRL + A to select all of the templates in the list. In this case there is no restriction at all and an experiment can be created on every page reference.

Initially, when the project component is configured, no template is selected. Due to this, the name of the list is displayed in red and it is not permissible to carry out experiments. The Create experiment button or menu item is thus hidden in both clients until at least one template is selected.

Deleting pages

Next, the user must decide what happens to the pages of the individual variants when variants are deleted from a current experiment or when an experiment is finished. The corresponding checkbox Delete pages is deactivated by default. In this case, the pages of all variants are retained following the removal of a variant from a current experiment or after an experiment finishes. If the checkbox is activated, the pages of the variants that are not being used are deleted.

If the initial original page contains more references to other page references, it will be retained regardless. This still applies even if checkbox Delete pages is activated and the page is not selected as the winning variant. Otherwise the other page references would no longer contain an associated page and if they were to be called, this would lead to exceptions.

The page references of the variants that are not being used and those from the dispatcher are always deleted, regardless of the status of the checkbox.

Importing templates

Finally, the various templates can be transferred to the project with the Import templates button. They provide the functions of the A/B-Testing module to the editor.

4.3. Activating the web component

A web component must be added to the project. To add a web component, open the ServerManager and select Project propertiesWeb components.

Web components in the project properties
Figure 8. Web components in the project properties


Inside the main panel, various tabs are visible, which contain a list of the existing web components. Select all the tabs in succession and click Add. Next, select the A/B-Testing WebApp and click OK to add it. The web component is added to the list in the main panel (see figure Web components in the project properties).

The web component must be installed on an active web server and then activated. The server can be selected using the selection box.

The Active web server must be a web server with a servlet engine, version 3.0 or higher, and JSTL version 1.0 or higher.

More detailed information on how to add web components is available in the FirstSpirit Documentation for Administrators.

4.4. Specifying the metadata template

When different variants are created for an experiment, the associated dispatcher page reference is saved with them. This assignment is made using the metadata.

The template must be specified in the project properties and has to exist in the project for this purpose. If the template does not exist, start by creating it. If the template already exists, open the ServerManager and select Project propertiesOptions. Then, click the corresponding button to select the metadata template (see figure Options in the project properties). Click OK to save the changes you have made.

Options in the project properties
Figure 9. Options in the project properties


4.5. Analytics tool

Tracking during the runtime of an experiment is essential in order to analyze the success of each individual variant. Tracking is not part of the A/B-Testing module and must be carried out on a project-by-project basis. An analytics tool is required for this purpose (users are essentially free to choose which one). In this regard, the A/B-Testing module has a completely open architecture and is not limited to specific services.

The Google Analytics plug-in was added to the project with the import completed during the configuration of the project component. As its name suggests, this tracking plug-in is designed for Google Analytics and, therefore, requires a corresponding account.

If you are already working with another analytics tool, you should consider implementing your own plug-in for tracking purposes. Using the Google Analytics function is only recommended if you also use Google Analytics for other purposes. Otherwise, you should use a different analytics tool.

Please refer to the supplier's documentation when configuring the analytics tool you have chosen to use.

5. Adaptations in the FirstSpirit project

Once the various components have been installed and configured and an analytics tool has been registered, a number of adaptations must be made in the project you are using. The steps to be completed are described in the following sub-chapters.

5.1. Expansion of the metadata template

An experiment consists of a dispatcher (created automatically) and any number of variants. Each variant must be assigned to the dispatcher page of the corresponding experiment. This assignment is made with the following input component:

Metadata input component. 

<CMS_INPUT_TEXT name="md_experiment_uid" hFill="yes" singleLine="no" useLanguages="yes" hidden="yes">
	<LANGINFOS>
		<LANGINFO lang="*" label="Dispatcher" description="Dispatcher"/>
	</LANGINFOS>
</CMS_INPUT_TEXT>

The input component is hidden. It must be added to the metadata template of the project you are using. The UID of the dispatcher page reference of the experiment is written to it automatically when a new variant is created.

If a metadata template does not already exist in the project, one must be created.

Select the metadata template in the project properties.

More information about metadata is available in the FirstSpirit Documentation for Administrators.

5.2. Expansion of the page template

The functions of the A/B-Testing module are provided via four format templates. These templates must be referenced in the HTML code of all permissible page templates. The A/B-Testing Head, A/B-Testing Body, and Traffic Allocation Plug-in templates were added to the FirstSpirit project during the import completed when the project component was configured. The fourth template corresponds to the tracking plug-in to be integrated.

A/B-Testing Head
This format template registers the integrated plug-ins and incorporates the CSS for the A/B-Testing bar in the preview.
A/B-Testing Body
This format template takes the tracking code from the integrated tracking plug-in and makes it available by referencing the A/B-Testing Tracking format template. It is also needed to show and hide the A/B-Testing bar.
Traffic allocation plug-in
The traffic allocation plug-in makes the settings dialog of the A/B-Testing bar available. The participation rate for the entire experiment and the distributions of the individual variants are defined and saved in this dialog.
Tracking plug-in
If you do not wish to use the Google Analytics plug-in supplied with the module, the tracking plug-in must be implemented on a project-by-project basis. Among other things, the plug-in contains the necessary tracking code which enables the success of the variants involved in an experiment to be determined. It is also used to query other specific data that might be available.

All four format templates are essential to the use of the A/B-Testing functions. They must be referenced in the page template you are using with CMS_RENDER calls:

Referencing format templates in the page template. 

<head>
   [...]
   $CMS_RENDER(script:"has_experiment", pageref: #global.node)$
   $CMS_IF(hasExperiment)$
      $CMS_RENDER(template:"abtesting_head")$
      $CMS_RENDER(template:"traffic_allocation_plugin")$
      $CMS_RENDER(template:"TRACKING PLUGIN REFERENCENAME")$ ❶
   $CMS_END_IF$
</head>
<body>
   $CMS_RENDER(template:"abtesting_body")$
   [...]
</body>

The reference name of the tracking plug-in to be integrated must be entered here.

The JSTL tag library below and the JSP page directive must be integrated in addition to the format templates. However, unlike the format templates, both calls should be placed at the start of the HTML code.

If this is a JSP project, the lines may already be there.

JSTL tag library and JSP page directive. 

<%@ page language="java" contentType="text/html; charset=$CMS_VALUE(#global.encoding)$" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

It must also be ensured that all variants involved in an experiment are JSP pages. To make sure of this, the file extension jsp must be defined in the Properties of the corresponding page template (see figure Defining a file extension).

Defining a file extension
Figure 10. Defining a file extension


5.3. Permission definition

It is essential for editorial permissions to be in place in order to execute the various functions supported by the A/B-Testing module. The table below explains which permissions are needed during the runtime of the experiment and which function belongs to each permission.

PermissionContentSiteExplanation

Visible/Read

The ability to view objects and read their content is required as standard.

Change

Various steps in an experiment need the ability to change objects:

while an experiment is being both created and finished, the URL switches between the original page and the dispatcher. This changes both pages.

What's more, a two-way link is created between every new variant and the dispatcher. This step requires a change to be made to the dispatcher too.

The need for the Change permission is self-explanatory when it comes to editing existing variants.

Create object

The dispatcher and the first variant are created automatically when an experiment is created. Each additional variant must be added manually. In both cases, the permission to create objects is required.

Create folder

This permission is not needed to carry out experiments.

Remove object

When an experiment finishes, the project is purged of the page references and, depending on the configuration, of the pages of the discarded variants and the dispatcher.

The same applies to removing variants from an existing experiment.

The permission to delete objects is indispensable for these actions.

Remove folder

No folders are deleted in connection with an experiment.

Release

Various objects are released when an experiment starts, finishes, and is continued.

Show metadata

The visibility of metadata is fundamental when carrying out an experiment. If it is not visible, an experiment cannot be started, continued, updated or finished. The view permission is also needed to edit settings.

Change metadata

When an experiment is created, as well as new variants, a two-way link is generated between the variants and the associated dispatcher. Since the link is stored in the metadata of the respective page references, it must be possible to change that data.

Change permissions

This permission is not needed to carry out experiments.

Select the root node of the PageStore or the SiteStore to distribute the editorial permissions. Then, right-click to open the context menu, select ExtrasChange permissions, then open Permission assignment. Assign the relevant permissions to the editors of your project in the dialog that appears.

When an experiment finishes, the page references and, depending on the configuration, the pages of the discarded variants and the dispatcher are removed from the project. The URL of the dispatcher is also transferred to the new original page.

These changes are persisted in the PageStore or the SiteStore by releasing the page and the page reference, as well as the respective parent folder. This is why the release permission is indispensable explicitly for finishing an experiment too.

5.4. Release workflow

The workflow selected in the project component is executed on the dispatcher when carrying out an experiment via the A/B-Testing bar. However, it can also be carried out manually on the individual variants using the general FirstSpirit functions. In both cases, it is essential to release all pages and page references involved in the experiment in order to avoid errors during generation and when live.

The module is supplied with an example workflow which meets this requirement. However, it is also possible to adapt an existing project-specific workflow. Both options are full-fledged alternatives. They are described in the following sub-chapters.

Only one workflow at a time can be executed per experiment. This means:

  • The workflow selected in the project component cannot be executed via the A/B-Testing bar if one of the variants is locked or already involved in a workflow.
  • Manual execution of a workflow on a variant is not possible if the page or page reference of the associated dispatcher is already involved in a workflow.

In both cases, the editor is notified accordingly and the corresponding workflow cannot be started.

5.4.1. API

When an experiment is carried out, all of the pages and page references involved must always be released. If a project-specific workflow was selected during the configuration of the project component, its function must be extended accordingly. The A/B-Testing API provides a number of methods for doing this; these are described briefly below. The abtesting-api folder containing the Javadoc documentation is also included in the scope of supply.

isDispatcher
This method determines whether the passed page reference is the dispatcher of an experiment. It returns true or false.
hasExperiment
This method can be used to check whether a page reference is involved in an experiment. If the method returns true, the page reference is a variant.
getVariants
This method fetches all variants of an experiment whenever the associated dispatcher is passed to it. If it fetches another variant or a page reference which cannot be assigned to an experiment, the list returned will be empty.
findDispatcherPageRef / findDispatcherPageRefForVariant
These two methods support the same function and differ only with regard to the parameter that is passed. In the case of the former method, the Uid serves as the basis for finding the corresponding dispatcher. In the case of the latter method, the page reference of a variant serves as the basis for finding the corresponding dispatcher.
loadPageRefByUid
This method uses a passed Uid to return the associated page reference.
isDispatcherInWorkflow
This method checks for a transferred variant whether the page or page reference of the associated dispatcher is already involved in a workflow. If it is, the method returns true and the execution of the workflow started manually on the variant must be canceled. If it was not canceled, status displays might be incorrect and there may be inconsistencies after going live.
isRemainingVariant
At the end of an experiment, the workflow selected in the project component is executed on the winning variant. It must be ensured (in this case only) that the workflow runs automatically and the editor has no means of canceling it. Otherwise this can lead to inconsistencies. Therefore, this method can be used to check whether a passed page reference is the winner variant of an experiment that is to be finished. If it is, the method returns true.
finalizeExperiment
The metadata field added when the project was adapted is used when finishing an experiment. Once the winning variant has been selected and the experiment has been finished, the content of this field must be deleted. This option is supported by the method.

The methods can be used in a script that is to be added to an activity of the release workflow being used. It makes sense to select the first activity in this case.

5.4.2. Workflow supplied

The A/B-Testing module is supplied with a workflow which meets the requirements described. To use it, select it in the configuration dialog for the project component. Since this workflow is based on the BasicWorkflows, these are a prerequisite of the project.

Installing the BasicWorkflows module

If the BasicWorkflows are not already part of the project, before using the workflow supplied, you must install the BasicWorkflows module on the FirstSpirit Server and activate the web component. Proceed as described for installing the A/B-Testing module and activating the associated web component. However, the web component for the BasicWorkflows is only needed on the ContentCreator tab.

In addition, the workflow must be activated for ContentCreator by selecting the Element Status Provider supplied. To do this, open the ServerManager and select Project propertiesContentCreator. Then change the Element Status Providers entry to the BasicWorkflows Status Provider entry and confirm the change by clicking OK.

Selecting the Element Status Provider
Figure 11. Selecting the Element Status Provider


Importing scripts and the workflow

Next, the BasicWorkflows scripts used for the release workflow must be added to the project. To do this, go to the Template Store in SiteArchitect and click to select Import from the context menu. This opens an import dialog box where you should select the export_basic_release import file from the workstation directories. Click Open to confirm; a second dialog is displayed listing all of the elements contained in the file. Since only the scripts are needed, the import of the release workflow can be deactivated (see figure Importing scripts). Finally, repeat the same process to import the workflow supplied with the A/B-Testing module.

Importing scripts
Figure 12. Importing scripts


The imported workflow must now be authorized in the individual stores so that it can be executed on FirstSpirit elements. To do this, select Change permissions from the context menu of the stores' root nodes to call up the permission assignment. Next, on the Workflow permissions tab, activate the Authorized and Use release permission checkboxes for the imported workflow. To finish, click the OK button to close the dialog.

Functionality

Once installation and import processes are complete, the workflow can be used in the project. The workflow can essentially be executed on various elements:

Dispatcher
When an experiment is started, updated, and continued via the A/B-Testing bar, the workflow is executed on the associated dispatcher page. In this case, both the dispatcher and all of the variants associated with the element are released automatically.
Variant
The general FirstSpirit function can also be used to execute the workflow on a variant. In this case, the editor must confirm a security prompt in order to release the associated experiment. If the editor confirms this prompt, all variants and the dispatcher are released. If the editor does not confirm this prompt, the workflow is canceled and the variant remains in its previous status.

Only one workflow can ever be executed per experiment. This means:

  • The workflow selected in the project component cannot be executed via the A/B-Testing bar if one of the variants is locked or already involved in a workflow.
  • Manual execution of a workflow on a variant is not possible if the page or page reference of the associated dispatcher is already involved in a workflow.

In both cases, the editor is notified accordingly and the corresponding workflow cannot be started.

If the workflow is executed manually using the general FirstSpirit function, it is necessary to also manually reload the preview in order to update the status of the A/B-Testing bar.

Element without reference to an experiment
If the workflow is executed on a FirstSpirit element that is not involved in an experiment, it will behave as per its standard functionality.

You can find more detailed information on the BasicWorkflows in the associated documentation.

6. Tracking

An experiment is only worthwhile if it returns a meaningful result. This depends in turn on the success of the variants. Tracking during the runtime of the experiment is essential in order to analyze the success of each individual variant.

Tracking is not part of the A/B-Testing module and must be carried out on a project-by-project basis. In addition to an analytics tool that can be freely selected by the user, a tracking plug-in is required in the project for this purpose. The plug-in corresponds to a format template and must contain the tracking code, among other things.

The Google Analytics plug-in was added to the project with the import completed during the configuration of the project component. In order to use it, you must complete just a few configuration steps which are described in the next sub-chapter. You should only consider using the plug-in if you are also using Google Analytics for other purposes.

Otherwise, we recommend choosing a different analytics tool. In this case, a project-specific tracking plug-in is required. Developers can implement this plug-in easily. The requirements to be met are also described below.

6.1. Google Analytics plug-in

The Google Analytics plug-in is supplied with the module. Designed for Google Analytics, as its name suggests, the plug-in is added to the project with the import completed during the configuration of the project component.

In addition to having a Google Analytics account, you simply need to add some information to the project settings template and select it in the project properties. It needs to be referenced in the page template too.

The plug-in also requires a Google Analytics Id and a Dimension Id:

All of the necessary steps are described below.

6.1.1. Registration and configuration of a Google Analytics account

A Google Analytics account is required in order to use the Google Analytics plug-in that is supplied with the module. If you do not have an account, you can register here: http://www.google.com/analytics/

A wizard guides you through the steps of the registration process that must be completed in order to create an account.

Registration of a Google Analytics account
Figure 13. Registration of a Google Analytics account


Confirm the Google Analytics terms of use to complete the registration process.

More information about configuration is available in the Google Analytics help.

6.1.2. Plug-in integration

The Google Analytics plug-in is imported into the project during the configuration of the project component. It is made available as a format template and contains the tracking code for capturing the success of the variant. It also serves to query the Dimension Id, which must be entered when configuring an experiment.

In order to use the Google Analytics plug-in, it must be integrated in all permissible page templates by means of a $CMS_RENDER$ call. It is important to note that the call must be positioned in the page header after the reference to the A/B-Testing Head.

Referencing the Google Analytics plug-in in the page template. 

<head>
   [...]
   $CMS_RENDER(script:"has_experiment", pageref: #global.node)$
   $CMS_IF(hasExperiment)$
      $CMS_RENDER(template:"abtesting_head")$
      $CMS_RENDER(template:"traffic_allocation_plugin")$
      $CMS_RENDER(template:"google_analytics_plugin")$
   $CMS_END_IF$
</head>
<body>
   $CMS_RENDER(template:"abtesting_body")$
   [...]
</body>

6.1.3. Expansion of the project settings template

The Google Analytics plug-in supplied with the module uses the following input component, which must be added to the project settings template. If a project settings template does not already exist in the project, one must be created. The template must also be selected in the project properties.

Project settings input component. 

<CMS_GROUP>
   <LANGINFOS>
      <LANGINFO lang="*" label="A/B-Testing"/>
   </LANGINFOS>

   <CMS_INPUT_TEXT name="ab_googleid" hFill="yes" singleLine="no" useLanguages="no">
      <LANGINFOS>
         <LANGINFO lang="*" label="Google Analytics Id"/>
      </LANGINFOS>
   </CMS_INPUT_TEXT>
</CMS_GROUP>

Within the project, the Google Analytics Id must be added to the project settings by entering it in this field. The Id is used by the plug-in. It enables the variants involved in an experiment to be tracked by Google Analytics.

You will find the Google Analytics Id in your Google Analytics account on the following path: AdminAccountPropertyTracking InfoTracking Code.

Path to find the Google Analytics Id
Figure 14. Path to find the Google Analytics Id


6.1.4. Specification of the project settings template

If you are using the Google Analytics plug-in, in addition to the metadata template, the project settings template also has to be added to the project properties. The template is used for the provision of the Google Analytics Id; it must exist in the project and be expanded as described in the previous chapter. If the template does not exist, complete these steps first.

If the template already exists, open the ServerManager and select Project propertiesOptions. Then click the corresponding button to select the project settings (see figure Options in the project properties). Click OK to save the changes you have made.