Enhance Your Email Marketing Campaigns with Multivariate Testing

If you are running a business, you know you have to be competitive at all times. One way to achieve that is to become creative, especially when trying to pick up sales and improving email marketing campaigns. It is difficult because you will never know what will be the outcome of all your efforts. However, you can optimize and eventually enhance your promotion through testing.

Let us say you are running a fancy ice cream shop. Customers would take a look at different flavors, and most of them would wish they could taste them all. Finally, they have narrowed down their choice to two (or three or even four) flavors.

Then, much to their joy, you ask them, “Would you like to try any flavor?” Tasting the ice cream can help your customers find the ones that best suit their moods and preferences.

That scenario is just like with email testing. It is essential that you can make an informed decision by knowing and testing every option you have.

However, things can become complicated. With one scoop, the choices are straightforward. What happens though when there are more scoops to follow? Your customers may love coffee and blue moon flavors but will those two go well together? The answer is no.

Getting two or more scoops of ice cream is often not as easy as most people think. It takes time to master the perfect combination – and requires an awful lot of tests.

Just like with figuring out what will work together, email marketing campaigns need to be tested as well. You probably heard about A/B, also called A/X testing where you have two or more versions of your email that you will send to your customers. In each version, you change one variable, such as the From name, the subject line, or the call to actions. It is very much like selecting the best flavor that will work for your customers.

As previously mentioned, you have to be creative with your email marketing campaigns. This is why you should step up your game, go for at least two scoops, and add a topping. You can make your email marketing strategy even tastier with the help of multivariate testing (MVT).

Multivariate Testing: What It Is and How It Differs from A/B Testing

To make things simpler, A/B testing is where you duplicate your email template. Then, you just take one element and change it. It is sometimes called split testing where you find out which version yields the most number of clicks or opens.

As for multivariate testing, you have two or more elements in focus, and you analyze how the combinations perform. It is more than A/B, as it is A/B/C…/Z test.

For example, in A/B testing, you test two versions of your email in which the subject lines differ. For multivariate testing, you have multiple subject lines tested against each other.

Unlike A/B testing, multivariate testing isolates several small items of the email so that the marketer understands the individual effect of those elements on the conversion rate. It can also identify compound effects based on the interaction of the customers between independent factors.

Let us say that you have a campaign whose aim is to boost your sales. In addition, the email should encourage more people to click through your landing page. In this case, you can test:

The call to action text: You either use “Buy now” or “Find out more about our amazing prices!”

The call to action position: You want to know whether you should add the CTA at the bottom of the email or before the fold.

Through a multivariate test, you will create four versions of your email (2 CTA texts x 2 CTA positions). It does seem easy to test, but it becomes more complicated as you add one more element. However, the results will be even more effective and powerful.

As you test the CTA text and position, you can also check whether the color of the text will work or not. To start testing, there are applications and services like MailChimp that allow multivariate testing. Instead of just four email versions, you will now have to make eight (2 CTA texts 2 CTA positions x 2 CTA colors). That may be equal to six, but the total is actually eight. Here’s why:

  1. “Buy now” CTA before the fold in red.
  2. “Buy now” CTA before the fold in black
  3. “Buy now” CTA at the end of the email in red.
  4. “Buy now” CTA at the end of the email in black.
  5. “Find out more about our amazing prices” CTA before the fold in red.
  6. “Find out more about our amazing prices” CTA before the fold in black.
  7. “Find out more about our amazing prices” CTA at the end of the email in red.
  8. “Find out more about our amazing prices” CTA at the end of the email in black.

With the variables ready, you can use your multivariate testing software and analyze the results. You will see the CTA combinations that can result in more clicks.

Ground Rules for Testing

Multivariate testing has no limits when it comes to testing the email elements you want. Therefore, it is easy to get carried away, which can lead to analyzing some crazy options. Although you can check whatever you want, you also don’t want to end up wasting your time. To make it simpler for you, here are some tips to follow:

Know why you are testing.

While you can check several elements for your next email campaign, the tests are useful only if you can determine what the problem is. For example, you are not getting as much website traffic as you predicted. Testing items, such as your CTAs, can help you solve the issue in bringing more clicks to your landing page.

Consider the elements that should be tested.

For instance, your open rate is 35%, but your clickthrough rates (CTRs) are still small, this does not call for you to test your “from” name or the subject line. Instead, you should test the things your contacts see as they open your email. For example, you should find out if the email heading is engaging or if the CTAs are easy enough to locate.

Don’t forget about your metrics.

While email stats give you the information you need after the tests, it is more important to know the results that really matter to you. Subject lines, from names, and the like will determine your open rates. Meanwhile, CTA texts, colors, and placements will give you your CTR.

Don’t waste time.

You can compare several disparities of your email but as the saying goes, time is money. When you add more elements, you will need to create more variations. Therefore, it will take you longer to obtain conclusive results. If it is truly needed to test several variations, you can try pairing similar elements, such as headline texts and photos or button placement or color.

With multivariate testing, you can fine-tune your email campaigns better than A/B testing. Often in email marketing, small changes can have a huge impact. You are not limited to just two email items at the same time because you can test as many as you want. If you know how to use multivariate testing correctly, you can tweak your email marketing experience and get the best possible result.

Multivariate Testing for Emails in Real Life

You have approximately seven seconds to grab and hold the attention of your readers. Email marketers know how important it is to make snap decisions based on everything that can be used and done in the first few seconds. There needs to be a good impression with the whole campaign, whether it is to make more sales, encourage new members to sign up, or to generate more downloads.

Here are some scenarios in which multivariate testing can help you with your campaign and your business:

Enticing subscribers to notice product offer and getting them to act on it.

A certain email has all the right ingredients for a piece of software: a good product name, description, high ratings, awards, testimonials, and has a noticeable download link. However, only 40% of the recipients downloaded the software, which was also easily accessible and free as well. So what happened to the other 60%?

Using MVT, this challenge can be fixed. In this case, the goal is to have the recipients download the software, while the issue that should be solved is the low rate of downloads. There is at least one reason why subscribers did not download, which may include the content of the email is not compelling enough, not tracking customers’ behavior and history toward the product, service, or website, and the prominence of CTAs.

Therefore, one test is for the CTA text for downloading the software:

  • “Download”
  • “Download for free”
  • “Download now”
  • “Instant Download”

In line with the text, you can add the size of the heading and the color used. A total of 12 versions (4×3) were created, each with a combination of the different headings for the download text and links for the download.

To determine which among the combinations work, the number of downloads is tracked. The test was run for a month, and the results were in: the word “Download for free” in red and large font size pushed the download rate from 40% to 63%.

What the email marketer learned about is that the word “free” can powerfully grab attention. If it is not super-obvious on your email, you are not doing a good job in your campaign. Additionally, the best location for the free offer is near the call to action.

Increasing the number of new members by measuring the clicks on the green sign up button.

For this test, the marketer ran the test for 4,000 email recipients. The original email had the headline saying to start an account. The bold 30-day free trial was in the center of the subhead. The rest of the subhead had the words “Pay as you go” and “No hidden fees” in another color.

When different versions of the email were tested, it was discovered that the original email was the worst performer. The winner of the test showed that conversion could be improved up to 30% compared to the first email.

The results presented that there should be even more emphasis on the 30-day free trial, which they should make the headline. The subhead should let people know that signing up for an account is quick and only takes less than 60 seconds. Also part of the subhead is a sentence telling people to pick a plan. In the tests, it was the only one with an exclamation point.

What the marketers learned about this test is that the word free should be emphasized and the heading should contain the duration of the free trial. Most people become interested in long trial periods but instead of saying one month, “30-day free trial” is used. Additionally, the exclamation point attracted more signups as it may exude more emotion for some viewers.

Getting more people to order.

This email marketing campaign showed the marketers that just one simple change could make splashes. They tested an email encouraging the readers to click on the link in the message to order. Based on the tests, they discovered that by simply modifying the “globe” image to a “lock.”

There were a total of 16 combinations for testing the headings and subheadings. After a few weeks of running the test, only eight of the variations were left to see which among the combinations consistently did great.

After the tests, it was discovered that the best performing combination has an image of a lock. It performed two to three times better than the original email campaign with a globe. Perhaps the biggest reason behind this result is that the lock presents security for people interested in making an order. Since the campaign was for a hosting service, customers want a secure, reliable, and safe hosting option, which the “lock” may represent for them.

Every email campaign is unique, and conversion goals are different. While some things may work for many people, such as the word “free,” it does not necessarily mean your campaign will perform the same. Setting up a multivariate test will help determine the effectiveness of your email strategy.

Key Takeaways

  1. Multivariate testing lets you evaluate different combinations of variables at the same time.
  2. It can give you valuable insights into your audience’s preferences that will help your future email marketing.
  3. To do multivariate testing, choose variables to form hypotheses, have a large email list, and know how to interpret your results.
I’m the Software Nerd who loves to grow online businesses. Unlike the typical marketer, I run experiments and bend computers to my will to achieve my results. Learn from my successes and failures.