How to A/B test flow branches

You will learn

Learn how you can A/B test flow branches using the conditional split component to test things like timing, discounts, and number of emails within a flow. In this article, we run through how to set up and end an A/B test for a flow branch. 

You can also A/B test a single flow email. Read how to A/B test flow emails for guidance.

Set up a test branch

1. Navigate to the flow you would like to A/B test in the Flows tab.
2. Drag a conditional split to where you would like to start your A/B test.
3. When configuring the split, select Random sample as the condition. Here, you will be prompted to select what percentage of your audience will go down the YES path, which you can think of as the control.
4. For an even A/B test, select 50%. Otherwise, click the dropdown to choose a different percentage.

The split will randomly choose a branch per profile. Because this is completely random, you may not see a completely even amount of profiles in each branch with a 50% split, but it will be close.

For example, you may want to test sending an additional email to one branch of your welcome series. To do this, you would drag a split to the end of your flow, add a time delay, and then configure the email content.

Once you have your split configured, you can build out the content of your test branch. Depending on what you're testing, this could be several emails or different timing from what you have in your control branch.

When running an A/B test, bear in mind that testing more than one variable at a time can skew results and make it difficult to determine how to attribute any differences in revenue, open rates, etc. For this reason, it's a best practice to test variables one at a time after you've determined a winner. 

Apple Mail Privacy Protection (MPP), which was released with iOS15 and updates to other Apple devices, may lead to inflated open rates due to changes in how we receive open rate data.

If you are triggering flows off of opens themselves, we suggest creating a custom report that includes an MPP property to review these affected opens. You can also identify these opens in your individual subscriber segments.

Determine the best branch

To determine which time delay is best, review the analytics for each message. Depending on the flow, decide which branch is best based on the open rate, click rate, and conversion rate.

To quickly view the flow analytics for messages in your flow:

1. Click Show analytics in the top right of the flow builder.
   

2. Review the metrics of the messages in each flow branch and decide which branch you prefer.
   

Learn more from our article on understanding flow analytics.

End the A/B test

After deciding which branch is better for your audience:

1. Click on the conditional split.
2. To have all of the recipients go down the Yes path, set the percentage to 100% (as shown in the example below); If you want everyone to go down the No path, set it to 0%. This way, those who were already in the Waiting queue still receive the old message rather than being taken out of the flow like they would if you delete the branch.
3. In whichever branch is no longer in use, set the messages to Draft.

Additional resources

• How to A/B test a flow email
• Understand best practices for A/B testing
• Mastering Email A/B Testing: Klaviyo’s Proven Tips & Tricks to Boost Revenue