Split testing subjects in Sequences for ConvertKit

Split Test Sequences in ConvertKit

In ConvertKit, you can split test your subjects in every single broadcast. However, in sequences, you cannot.

This has popped up before in my clients’ requests as well as around the web, including inside the ConvertKit Facebook group.

I’m not sure about how other platforms handle it, but for ConvertKit, Nathan explains why this is the case and how he responds to this question is one of the reasons I love and enjoy working with ConvertKit and will recommend it.

Nathan in response to having Split Testing in Sequences inside ConvertKit

Without putting words in Nathan’s mouth here, I really love that he filters the features of ConvertKit based on what’s good for the wider audience.

On the flipside

For those of you who have clients and run businesses that are in the minority though, this is where the power of the platform ConvertKit is built on lies.

It also gives you the ability to test a variety of things, not just subjects.

Let’s lay the groundwork here first though.

For you folks from the town of Hurryville

If you want to just move forward and copy this into your account, click here and everything explained below will be created for you.

Step 1: Set up your sequences

Have version A of your sequence all set up and ready to go as a version one.

Then duplicate the sequence by going into the sequence, clicking on Settings at the top.

Then find “Duplicate This Sequence” and click that button.

Rename it to your version B and now change your subjects to the second variation.

Once you’ve completed that, head over to a Visual Automation.

Step 2: Time to set up your Split Test (or A/B Test)

Whether you want to do this in an existing automation or a new one, the setup is the same.

You will use a custom field to house the version of your test for that subscriber.

PRO TIP: For email automation platforms, use a prefix before your variables that you use for internal (non-displaying, front facing) data. This will keep your sanity later on the custom fields for a subscriber record.

Click the “+” (plus) to add a new node.

Select Action and then Set a custom field.

Choose Action based on Setting a Custom Field

Then you will want to either choose or create a new custom field called var_bucket and set the value to {{'now' | date: '%s' | modulo:2}}

What is happening here is the coin flip. By grabbing to time in seconds, which is what the 'now | date: '%s' is doing, and then taking the modulo of that number by 2 (which is the mathematical operation between two numbers to get the remainder), you will either have 0 or 1 which is then stored on the subscriber’s record in the var_bucket custom field.

Then add another node, this time selecting the Condition based on a Custom Field.

Condition based on a Custom Field on a Subscriber's Record

Your condition then is based on if var_bucket matches 0.

Condition for the Split Test Choice

Now it’s a matter of putting the subscriber down the path to the correct sequence. Version A if Yes, and Version B if No.

Split Test Sequences in ConvertKit

What’s nice and powerful about this, is that over time and with enough of a sample size, or as the experts say, statistical significance, you will see not just the open rate, but also the click-through rate, and unsubscribe rate side-by-side without diving deep into some report.

It’s also easy once you’ve decided which sequence wins. It’s easy to tweak the automation to just use the winning version.

Bonus Benefits

I don’t want to turn this article into how to run split tests (or A/B tests), but keep in mind that you want to test one thing, not all things.

One test would be the same content, but different subject lines. This would result in a difference in open rates (in theory).

Subject line testing is what most platforms allow you to test and that’s it.

By taking the approach above, you can then test other things one at a time, like the content style, or a call-to-action, or images vs no images, or the list is endless.

If you know that you have enough volume going through, the power of these test could significantly improve your email conversions.

/ Jason Resnick

Jason is a WordPress developer helping small businesses, design and marketing agencies achieve their goals by specializing in Ecommerce and increasing conversions. Learn more about him here.

More Posts | Twitter | Work ON Your Business Challenge | Coaching

One response to “Split testing subjects in Sequences for ConvertKit

  1. Hello,
    I went ahead and used the link to just import the copied version into my automation’s (thank you for that btw). I am a little confused about the tag though. Should I have a custom tag, or use the one you created? I am not a developer, so this can be slightly daunting for someone like me who is not as tech-savvy as someone like yourself. The sequences I have all set and ready to go, but I have a feeling I need the tag specifically tailored to a custom tag. Thank you for your input Jason

Leave a Comment

Your email address will not be published. Required fields are marked *