Introduction to A/B testing

A/B testing also known as split testing helps you to send different variations of an email to a small set of subscribers in your list. When the email is received by your subscribers, you can decide based on the engagement metrics like the number of opens and click-throughs in your reports, which version of your email performed better. Once you know the winning version, you can send that version to the remaining subscribers of your list, so that you get the best results from your campaigns.

How to setup

Setting up A/B testing for your email is important, as, through this testing, you get to know what your customers loved about your email. Even a small change such as the color of your CTA button or sending time can affect the performance of your email campaign to a huge extent.

Here’s how you can start your A/B test

• Create your email.
• Upload your HTML, use an existing template or create your own template using drag and drop.
• Once you reach the preview screen, select the A/B testing tab.

A/B testing helps you to optimize your email campaigns for a specific metric like subject lines, open rates, click-throughs, email content, sender address, and the email send time. In Juvlon, you can create 2 or 3 versions of your email for A/B testing.

Now follow the steps below to set up your A/B test.

1. Select the parameter to conduct A/B Test on

• Subject line

When you want to optimize the open rate of your email campaign, select the subject line as the test type. The subject line is your first interaction with your subscribers, so you need to make sure that your subject line is compelling enough for your subscribers to open your email. Here are some tips for creating engaging subject lines.

• Email content

When you want to optimize the click rate of your email campaign, select email content as the test type. It deals with the email design, content, color of CTA, images, color scheme, location of images and CTAs. Play with your content and email design to find out what works best for your business. The version with high click rates is the one with engaging content and perfectly structured CTAs.

• Sender address

Sender address is your identity from which you have sent your email. Select the sender addresses through which you want to send your emails.

• Send time

Try testing different send times to know at what time you usually get good opens and click-throughs so that you know what time works best for your business.

2. Select the number of versions for A/B Test

Using Juvlon, you can create and test 2 or 3 variations of either subject line, email content, sender address or send time. Just select 2 or 3 while setting up your A/B test.

3. Select lists

Select the lists on which you want to conduct the A/B Test. To conduct an A/B test in Juvlon, you need to have at least a total of 900 subscribers in your lists.

4. Selecting the sample size

Choose the percentage of your subscribers that you want to participate in the A/B test. Each version of your email will be sent to the equal number of subscribers participating in the test.

Select a minimum of 10% of your total email subscribers from your list. The maximum subscribers that you can select to test is 50% of your total list. The remaining percentage of subscribers will receive the winning version of the email.

Note: The A/B testing procedure for the send time is a bit different.  If you’re testing for the Send time, you will always send emails to 100% of the subscribers in your selected lists. The subscribers will be divided equally based on the number of versions that you had decided.

5. Choosing the winning criteria

You can select the winning email version based on the following criteria

• By clicks- The email version which has the highest number of clicks wins.
• By opens- The email version which has the highest open-rate wins.
• Manual- You can select the winner manually by referring to the engagement metrics in the A/B test reports.

6. Setting the test duration

Select the time for which you want to run the A/B test from the dropdown beside the test duration. During the test duration, emails will only be sent to the email addresses that you have chosen to participate in the test. The remaining emails will receive the winning variation of your A/B test after the test duration is completed.

7. Changes in the email with respect to the type to test

Once you select the number of versions, decide what changes you will need to make in order to know what works best for your business.

• Subject line testing

If you selected the number of versions 2, you will be testing your email for two different subject lines.

Craft your subject lines in such a way that they stand out in your subscriber’s inbox.

• Email content testing

When you select this option, a new window will open, wherein you will be able to either create an email template using drag and drop or you will be able to upload a new template. This will depend on how you created the first original email for this campaign. You will be able to edit/change content for the number of versions you selected to test.

You can make changes in your email template like the content, call to action buttons, images, etc. We suggest that you make minimum changes in your email content so that after testing, while you are checking your engagement metrics, you know what worked for you. If you change a lot of parameters, you won’t be able to find out what exactly worked well for you.

• Sender address testing

If you selected the number of versions 2, you will be testing your email for two different sender addresses.

• Send time

The A/B test process for send time is a bit different than the other types.

Select the time interval between the emails sent. Your email list will be divided equally based on the number of versions that you have selected. Leaving the time interval that you have selected, the Emails will be sent to the complete list.

Then based on your engagement metrics, you can decide the time slot when your emails got maximum opens and click-throughs.

8. A/B test reports

When the A/B test is completed, you will be notified by an email saying that the test is complete and your report is available.

To check your test reports, you need to navigate to the left sidebar.

Go to Campaigns–> Select A/B test campaign-> Click on the email to see the reports.

Sending the winning email version to the remaining list

When an email version wins, based on your selected criteria and duration, you will be notified that a winner was chosen. You will be able to see the winning email version.

If you chose to determine the A/B test winning criteria based on the Open Rate or the Click Rate, your winning email version will be sent automatically to the remaining subscribers in your selected email lists.

But if you chose to manually determine the A/B test winning criteria, you will be notified when the test duration time is over. At that time, you need to choose your winner manually based on the engagement metrics.