Campaign Testing – 7 Things You Need to Know

Testing marketing campaigns should be an integral part of any digital marketing strategy. There are a lot of decisions to be made. Lots of moving parts. Even if you keep it simple, you should test your way to success.

But testing is hard. You need to know math and stuff.

Well, not really. Common sense will get you most of the way.

Here are seven things you should consider when testing.

1) Testing for the right reasons.

Don’t use testing as a surrogate for backbone,

“I’m not sure which direction is best so let’s test them both.”

Indecision is not a good reason to test. First, it suggests you have no opinion. If you don’t have an opinion, then marketing isn’t your field. Campaign Testing is often used to avoid conflict. If you can’t handle conflict, then marketing isn’t your field. Seeing a pattern here. Get some backbone and make a decision.

Use marketing campaign testing to improve your brand’s marketing and help you achieve your business goals.

2) Testing to make it better.

If your marketing is working, don’t succumb to boardroom boredom, don’t stop using what’s working. If it’s not broken, don’t fix it. I can assure you; your customers are not as bored with what you are doing as you are.

But you should always try to make it better. Brand campaign testing will help you determine how to improve your online marketing. For example, you can use testing to assess an email subject line and increase open rates, which is fantastic. You can also use testing to improve click-through rates. Use testing to ensure your landing pages are attracting the right customers. Would a different subject line, offer, or marketing channel produce better results? You won’t know without testing.

Test your marketing to make it better. This is a great reason to test.

For example, you can test your website copy. Does it support your best customer’s journey? Does your website copy engage them? This is another great thing to test.

The axiom for this is “beat the control.” Figure out what’s working, and then test different aspects of the campaign versus the control to see if a change results in more sales.

Use split testing if you are trying to decide between two versions. Basically, split your potential customers into two parts and test two email subject lines by sending a different line to each group.

Campaign Testing – Beat the control – 7 Things Small Businesses Need to Know Click To Tweet

3) Testing big things.

Testing things that matter, things that make a difference to your audience. Green versus blue isn’t a big thing. Don’t test this. Decide which is best. See “testing for the right reasons” above.

A different target audience or future campaigns, or will a new call to action, ‘Click Here’ versus ‘Learn More,’ improve the click-through rate?

Different versions of a landing page design, alternative headlines, or will a new offer create more traffic?

These are big things. These are necessary changes that matter to your audience.

Is the new offer or the new ad more effective with your target market?

Will a long or a short copy block deliver the benefits and drive more traffic?

Images?

Testing big things matters.

An email campaign is a fast way to test big things. An email campaign to your audience will quickly tell you what’s working and what’s not. Tests using a Facebook ad or an ad on another social media platform are another easy and relatively inexpensive way to assess variables.

Campaign testing that’s focused on big things will improve your marketing.

4) Testing things in isolation.

Be sure to isolate the element you are testing or, said another way, don’t test multiple things simultaneously. Leave multivariate testing to the professionals. Unless you are a very sophisticated mathematician who thoroughly understands multivariate testing using a large data set, you won’t know which element caused the success or failure.

Keep things simple. Don’t try multivariate tests. Pick one big thing and test it. Fine-tune the test and run it again. Then pick another big thing and test that. Don’t create problems for yourself.

 5) Consider the sample size.

Don’t test a larger sample than you need. Determine the sample size you need for statistically significant results. Work back up the response waterfall to determine the initial audience size. For example, if you need 200 sales to have a statistically valid comparison, then apply your conversion rate and response rate to 200 to determine the test cell sample size.

As a general rule, at least one hundred and up to 10% of your audience but no more than one thousand works. If your test cell is too large, you are needlessly putting valuable marketing resources at risk. If it’s too small, you will be making decisions based on inaccurate results. You don’t want to roll out a campaign based on test results that aren’t statistically significant. Neither outcome gets you where you need to be.

There are many other tools available to help with this. You don’t need to know the math. The folks at Optimizely have a handy calculator: A/B Sample Size Calculator. You can also review our post, Measuring Statistical Significance and Why It’s Important to Marketers, and use the embedded statistical significance calculator.

6) Testing what you can afford.

When you test, be sure that you run tests that will scale. Don’t test an offer you can’t afford to roll out or a media channel you can’t afford to use. Guess what? It might work. Then what?

If your offer does improve performance, but you can’t afford to make the offer at scale, you have put time and resources into testing something that you can’t use, which is pointless. Marketing campaign testing doesn’t help If you can’t afford to use the result in a full rollout. To know what you can afford, determine your allowable cost per order

7) Roll out your successes slowly.

Your landing page split test was a success. You got the desired outcome, and the results were statistically significant. Fantastic! “Let’s cancel everything and run with the new landing page.”

Not so fast.

Testing isn’t a perfect science. There may be other variables. Don’t join the New Coke Product Manager on the unemployment line. (I’ve been doing this for a long time, so you may need to Google this reference, it happened a long time ago.)

Before you cancel everything, confirm that the test reflects reality. Collecting data from a confirmation test will give you confidence that the change impacted the results. A confirmation test is usually the same as a successful test but uses a larger sample. Rolling it out is smart if your new landing page continues to perform during confirmation testing.

Marketing Campaign Testing – Conclusion

So have at it; test your way to marketing success.

Remember the seven keys to successful campaign testing:

1. Testing for the right reasons – don’t test to avoid making a decision
2. Testing to make it better – try to beat the control
3. Testing big things – test things that improve marketing effectiveness
4. Testing in isolation – multivariate testing is not practical
5. Use statistically valid sample sizes – don’t base decisions on flawed methodology
6. Testing what you can afford – test campaign variations that you can roll out
7. Roll successes out slowly – confirmation testing leads to conclusive results

The most important thing is don’t wing it; testing marketing campaigns brings better results.

Campaign Testing FAQs

A version of this post appeared in Red8 Interactive’s blog: 6 THINGS TO CONSIDER WHEN TESTING YOUR WEBSITE. Red8 is the team behind Inn8ly.