Split Campaigns

Owned by Ongage OngageConnect
Last updated: May 28, 2024 by Stepin\ Shashin Petel

Table of Contents:


Introduction

What are Split Campaigns

  • Split Campaigns (aka A/B Testing) allow email marketers to compare 2 and up to 10 versions of an email message or subject and see which version performs better.

How to Setup a Split Campaign

  1. Go to: Campaigns → A/B:

     
  2. There are few types of Split Campaigns you can choose from:
    1. Email/SMS – send different email message (2-10) to the same segment(s). The Emails will be spread evenly across all targeted recipients. In the case of A/B split half of the segment will receive Version A, while the other half will receive Version B.
    2. Subject – send different email subjects (2-10) to the selected campaign segment(s). The subject will be spread evenly across the entire target audience, and then you can see which performed the best.
    3. ESP – use the same Email Message, evenly spread across 2-10 different ESPs. 

Use a White-list segment to send all versions of a Split Campaign to stakeholders

Stakeholders and key staff member are often interested in seeing all the different versions sent to contacts in a Split campaign. In Sep 2018 Ongage enabled that by allowing marketers to select a White-list segment when configuring a Split campaign. All contacts belonging to that White-list segment segment will receive all versions of the Split Campaign. All other targeted segments will randomly receive just one of the versions from the split test.

  1. In the 'Edit' page of the Split Campaign, under the 'Choose Segments' dialog box, select the desired White-list segment that you'd like to send all versions of this A/B Split test to.


Using Email Tags to find contacts participated in certain email version of A/B Split

When setting up an A/B split test with two or more messages, follow these steps:

  1. Assign different tags to each message. See HERE on how to add Tag to Emails.
  2. Create a segment based on the tags assigned to the messages.
  3. Analyze which contacts received a message with “Tag A” versus those who received a message with “Tag B.”

Remember that you can reuse the same tags across multiple campaigns by using the “Sent from campaign with specific Tag” condition when creating segments. This approach ensures effective testing and optimization of your email campaigns.

For example if you want to see the number of contacts received email from campaign ID with certain email tag then you can create segment like below:  

Advanced Configuration

Real-Time aka Automatic A/B Split Test

  • In the advanced configuration section, of the split campaign, you can choose to set up a Real-Time (aka Automatic) Split-Test, that enables you to send a sample of your campaign, and then based on the results of that sample, it will send the rest of the campaign to the winning test.


Usage Recommendations

  • Sample set should be significant. Preferably between 15%-20% and typically no less than 10%.
  • Moreover the winner should be selected after a few hours, in order to allow for a statistically significant amount of open and click data to accumulate.

How does it work?

Let's say you set the sample size to be 10% for 2 a/b tests. Here's how it works:

  • 5% goes to group A
  • 5% goes to group B
  • 90% goes to group X (the undecided version which will be used for the winner).

According to those percentages, the first contact will go to group A, then the second one will go to group B, then the next 18 go to group X. Then it repeats: another one to A, another one to B, and another 18 to group X, In addition to this, there is a randomized offset to the starting group, so the result is, that the sample contacts for the A/B test is completely randomized.

Non-Even Split Tests

Typically in an A/B split test, group A gets 50% and group B the other 50%. But sometimes marketers for a variety of reasons may want group A to get 80% of split test and group B only 20%. For that, Ongage offers the custom split ratio. Here's how you can set it up:

  1. Go to the 'Advance configuration' of the Split Campaign, towards the bottom of the page, and check the 'Custom split ratio' option:


  2. There are two options available for choosing the message type as Email and SMS.


  3. The screen will automatically scroll back to the top of the page and additional input boxes to insert ratios will appear next to each test item. Set your ratios, so that they add up to 100%, and you're set! Here's an example of an non-even split test using email messages: