So you’re considering digital marketing. When you plan your digital marketing campaign, testing should be a consideration. There are a lot of decisions to be made. Lots of moving parts. Even if you keep it simple,¬†you should test your way to success.

But testing is hard. You need to know math and stuff.

Well not really. Common sense will get you most of the way.

Here are seven things you should consider when testing your marketing campaign.

1) Test for the right reason.

Don’t use testing as a surrogate for backbone:

“I’m not sure which direction is best so let’s test them both.”

Indecision is not a good reason to test. First, it suggests you have no opinion. If you don’t have an opinion, then marketing isn’t your field. Testing is often used to avoid conflict. If you can’t handle conflict, then marketing isn’t your field. Seeing a pattern here. Get some backbone and make a decision.

2) Test to make it better.

If your marketing is working don’t succumb to boardroom boredom, don’t stop using what’s working. If it’s not broken, don’t fix it. I can assure you; your customers are not as bored with what you are doing as you are.

But you should always try to make it better. Marketing campaign testing will help you determine how to make your online marketing better. For example, not only can you use testing to increase conversion, which is fantastic, you can use campaign testing to ensure you’re converting the right customers. Would a different message or offer or marketing channel produce better? You won’t know without testing.

Test your marketing to make it better. This is a great reason to test.

Beat the control.

3) Test big things.

Testing things that matter, that’s what makes a difference. Blue versus green isn’t a big thing. Don’t test this. Decide which is best. See “test for the right reasons” above. A new Call to Action or landing page design or headline, these are big things, so is a new offer or a new form. Long copy versus short copy? These are a big thing.

Email marketing campaign testing is a fast way to test big things, to determine what’s working and what’s not.

Test big things to make your marketing even better.

4) Test things in isolation.

Be sure to isolate the element you are testing or, said another way, don’t test multiple things simultaneously. Unless you are a very sophisticated mathematician who thoroughly understands multi-variant testing, and test design, you won’t know which element caused the success or failure.

Pick one big thing and test it.

 5) Consider the sample size.

Don’t test a larger sample than you need. Determine what sample you need for statistical validity. Work back up the response waterfall to determine the sample size. For example, if you need 200 sales to have a statistically valid comparison then apply your conversion rate and response rate to 200 to determine how many need to be in the test cell. If your test cell is too large, you are putting valuable marketing resources at risk needlessly. If it’s too small, you will be making decisions based on sketchy data. Neither outcome gets you where you need to be.


There are many tools available to help with this. You don’t need to know the math. The folks at Optimizely have a handy calculator: A/B Test Sample Size Calculator. Visual Website Optimizer also has a tool: A/B Split Test Significance Calculator. As you can tell from the very similar names, they do pretty much the same thing.

6) Test what you can afford.

When you test, be sure what you test will scale. Don’t test an offer you can’t afford to roll out, or a media channel you can’t afford to use. Guess what, it might work, but so what? If you can’t afford to use it for the full effort, don’t test it.

7) Roll out your successes slowly.

Your test was a success. Fantastic. “Let’s cancel everything and run with the new program.”

Not so fast.

Testing isn’t a perfect science. Don’t join the New Coke Product Manager on the unemployment line.

Before you cancel everything, confirm that the test reflects reality. Run a confirmation test. A confirmation test is usually the same as the successful test but uses a larger sample. If your new program continues to perform in the confirmation test, then rolling it out is the smart thing to do.

Conclusion – Digital Marketing Campaign Testing

So have at it, test your way to marketing success.

A version of this post appeared in Red8 Interactive’s blog: 6 THINGS TO CONSIDER WHEN TESTING YOUR WEBSITE. Red8 is the team behind Inn8ly.