Campaign Testing – 7 Things You Need to Know

Campaign testing should be an integral part of any digital marketing plan. There are a lot of decisions to be made. Lots of moving parts. Even if you keep it simple, you should test your way to success.

But testing is hard. You need to know math and stuff.

Well, not really. Common sense will get you most of the way.

Here are seven things you should consider when testing your marketing campaign.

1) Test for the right reason.

Don’t use testing as a surrogate for backbone,

“I’m not sure which direction is best so let’s test them both.”

Indecision is not a good reason to test. First, it suggests you have no opinion. If you don’t have an opinion, then marketing isn’t your field. Testing is often used to avoid conflict. If you can’t handle conflict, then marketing isn’t your field. Seeing a pattern here. Get some backbone and make a decision.

2) Test to make it better.

If your marketing is working don’t succumb to boardroom boredom, don’t stop using what’s working. If it’s not broken, don’t fix it. I can assure you; your customers are not as bored with what you are doing as you are.

But you should always try to make it better. Marketing campaign testing will help you determine how to make your online marketing better. For example, not only can you use testing to increase conversion, which is fantastic, you can use campaign testing to ensure you’re converting the right customers. Would a different message or offer or marketing channel produce better results? You won’t know without testing.

Test your marketing to make it better. This is a great reason to test.

For example, you can test your website copy. Does it support your best customer’s journey? Does your website copy engage them? This is another great thing to test.

The axiom for this is “beat the control.” Figure out what’s working, and then test other things to see if something else can work better.

Campaign Testing – Beat the control – 7 Things Small Businesses Need to Know Click To Tweet

3) Test big things.

Testing things that matter, that’s what makes a difference. Green versus blue isn’t a big thing. Don’t test this. Decide which is best. See “test for the right reasons” above.

A different target audience or call to action: Click Here versus Learn More.

A new landing page design or headline.

These are big things.

So is the new offer or the new form more effective? Long copy versus short copy? Images? Test big things.

Email marketing campaign testing is a fast way to test big things. An email to your customers will quickly tell you what’s working and what’s not. Facebook ads are another easy and relatively inexpensive way to test.

Marketing campaign testing that’s focused on big things will improve your efforts.

4) Test things in isolation.

Be sure to isolate the element you are testing or, said another way, don’t test multiple things simultaneously. Unless you are a very sophisticated mathematician who thoroughly understands multi-variant testing, and test design, and have large data sets you won’t know which element caused the success or failure.

Pick one big thing and test it. Then pick another big thing and test it.

 5) Consider the sample size.

Don’t test a larger sample than you need. Determine what sample you need for statistical validity. Work back up the response waterfall to determine the sample size. For example, if you need 200 sales to have a statistically valid comparison then apply your conversion rate and response rate to 200 to determine how many need to be in the test cell. If your test cell is too large, you are putting valuable marketing resources at risk needlessly. If it’s too small, you will be making decisions based on sketchy data. Neither outcome gets you where you need to be.

There are many tools available to help with this. You don’t need to know the math. The folks at Optimizely have a handy calculator: A/B Test Sample Size Calculator. You can also review our post, How to Measure Statistical Significance and Why It’s Important to Marketers, and use the embedded statistical significance calculator.

6) Test what you can afford.

When you test, be sure what you test will scale. Don’t test an offer you can’t afford to roll out or a media channel you can’t afford to use. Guess what? It might work, but so what? Marketing campaign testing doesn’t help If you can’t afford to use the result in a full rollout. To know what you can afford, determine your allowable cost per order.

7) Roll out your successes slowly.

Your test was a success. Fantastic. “Let’s cancel everything and run with the new program.”

Not so fast.

Testing isn’t a perfect science. Don’t join the New Coke Product Manager on the unemployment line. (I’ve been doing this for a long time so you may need to Google this reference, it happened a long time ago.)

Before you cancel everything, confirm that the test reflects reality. Run a confirmation test. A confirmation test is usually the same as a successful test but uses a larger sample. If your new program continues to perform in the confirmation test, then rolling it out is the smart thing to do.

Marketing Campaign Testing – Conclusion

So have at it; test your way to marketing success.

Remember the seven keys to successful campaign testing:

1. Test for the right reasons – don’t test to avoid making a decision
2. Test to make it better – test to beat the control
3. Test big things – test things that improve marketing effectiveness
4. Test in isolation – isolate what you are testing
5. Use statistically valid sample sizes – don’t base decisions on flawed methodology
6. Test what you can afford to roll out
7. Roll successes out slowly

A version of this post appeared in Red8 Interactive’s blog: 6 THINGS TO CONSIDER WHEN TESTING YOUR WEBSITE. Red8 is the team behind Inn8ly.