I’ve learned over the years that when launching a new product or service, it’s crucial to run a test campaign to gauge its potential success. A test campaign allows you to gather valuable data on consumer response, identify any potential issues, and make necessary adjustments before fully launching. By carefully planning and executing a test campaign, you can ensure that your final marketing strategy is well-informed and optimized for success. Whether it’s testing different ad creatives, targeting specific demographics, or experimenting with various messaging, a test campaign is an essential step in the marketing process.

Campaign testing should be an integral part of your digital marketing strategy. There are a lot of decisions to be made. Lots of moving parts. Even if you keep it simple, you should test your way to success.

But testing is hard. You need to know math and stuff.

Well, not really. Common sense will get you most of the way.

Here are seven things I’ve learned over my career that you should consider for better campaign testing.

1) Testing for the right reasons.

Don’t use testing as a surrogate for backbone,

“I’m not sure which direction is best so let’s test them both.”

Don’t Leave Empty Handed!

Get Digital Tips & Tricks Delivered to Your Inbox

  • This field is for validation purposes and should be left unchanged.

We will never share your email address.

Indecision is not a good reason to test. First, it suggests you have no opinion. If you don’t have an opinion, then marketing isn’t your field. Campaign Testing is often used to avoid conflict. If you can’t handle conflict, then marketing isn’t your field. Seeing a pattern here. Get some backbone and make a decision.

Use marketing campaign testing to improve your brand’s marketing and help you achieve your business goals.

2) Use campaign testing to make it better.

If your marketing is working, don’t succumb to boardroom boredom; don’t stop using what’s working. If it’s not broken, don’t fix it. I can assure you that your customers are not as bored with what you are doing as you are.

But you should always try to make it better. Brand campaign testing will help you determine how to improve your online marketing. For example, you can use testing to assess an email subject line and increase open rates, which is fantastic. You can also use testing to improve click-through rates. Use testing to ensure your landing pages are attracting the right customers. Would a different subject line, offer, or marketing channel produce better results? You won’t know without testing.

Use campaign testing to make your marketing better. This is a great reason to test.

For example, you can test your website copy. Does it support your best customer’s journeyDoes your website copy engage them? This is another great thing to test.

The axiom for this is “beat the control.” Figure out what’s working, and then test different aspects of the campaign versus the control to see if a change results in more sales.

Use split testing if you are trying to decide between two versions. Basically, split your potential customers into two parts and test two email subject lines by sending a different line to each group.

Campaign Testing – Beat the Control – 7 Things Marketers Need to Know Click To Tweet

3) Test big things.

campaign testing - test big things not green vs blue

Testing things that matter, things that make a difference to your audience. Green versus blue isn’t a big thing. Don’t test this. Decide which is best. See “testing for the right reasons” above.

A different target audience or future campaigns, or will a new call to action, ‘Click Here’ versus ‘Learn More,’ improve the click-through rate?

Different versions of a landing page design, alternative headlines, or will a new offer create more traffic? For more on split testing, read our post on the subject: Unleash the Power of Split Testing – Elevate Your Marketing.

These are big things. These are necessary changes that matter to your audience.

Is the new offer or the new ad more effective with your target market?

Will a long or a short copy block deliver the benefits and drive more traffic?


Testing big things matters.

An email campaign is a fast way to test big things. An email to your list will quickly tell you what’s working and what’s not. Tests using a Facebook ad or an ad on another social media platform are another easy and relatively inexpensive way to assess variables. For a deeper dive into email marketing, read our post: Email Mastery: How to Create a Newsletter in Mailchimp that Shines.

Campaign testing that’s focused on big things will improve your marketing.

4) Test things in isolation.

Isolate elements for better campaign testing

Be sure to isolate the element you are testing or, said another way, don’t test multiple things simultaneously. Leave multivariate testing to the professionals. Unless you are a very sophisticated mathematician who thoroughly understands multivariate testing using a large data set, you won’t know which element caused the success or failure.

Keep things simple. Pick one big thing and test it. Fine-tune the test and run it again. Then, pick another big thing and test that. Don’t create problems for yourself.

 5) Consider the sample size.

Don’t test a larger sample than you need. Determine the sample size you need for statistically significant results. Work back up the response waterfall to determine the initial audience size. For example, if you need 200 sales for a statistically valid comparison, then apply your conversion rate and response rate to 200 to determine the test cell sample size.

sample size matters to marketing campaign testing

As a general rule, at least one hundred and up to 10% of your audience, but no more than one thousand works. If your test cell is too large, you are needlessly putting valuable marketing resources at risk. If it’s too small, you will make decisions based on inaccurate results. You don’t want to roll out a campaign based on test results that aren’t statistically significant. Neither outcome gets you where you need to be.

There are many other tools available to help with this. You don’t need to know the math. The folks at Optimizely have a handy calculator: A/B Sample Size Calculator. You can also review our post, Measuring Statistical Significance and Why It’s Important to Marketers, and use the embedded statistical significance calculator.

6) Test what you can afford to roll out.

When you test, be sure that you run tests that will scale. Don’t test an offer you can’t afford to roll out or a media channel you can’t afford to use. Guess what? It might work. Then what?

If your offer does improve performance, but you can’t afford to make the offer at scale, you have put time and resources into testing something that you can’t use, which is pointless. Campaign testing doesn’t help If you can’t afford to use the result in a full rollout. To know what you can afford, determine your allowable cost per order

7) Roll out your successes slowly.

Your landing page split test was a success. You got the desired outcome, and the results were statistically significant. Fantastic! “Let’s cancel everything and run with the new landing page.”

Not so fast.

Testing isn’t a perfect science. (Only my wife is perfect.) Other variables may affect the test and its results. Don’t join the New Coke Product Manager on the unemployment line. (I’ve been doing this for a long time, so you may need to Google this reference; it happened a long time ago.)

Before you change what you are doing, confirm that the test reflects reality. Collecting data from a confirmation test will give you confidence that the change impacted the results. A confirmation test is usually the same as a successful test but uses a larger sample. Rolling it out is smart if your new landing page continues to perform during confirmation testing.

Marketing Campaign Testing – Conclusion

So, have at it; use campaign testing to find your way to marketing success.

Remember the seven keys to successful campaign testing:

  1. Testing for the right reasons – don’t test to avoid making a decision
  2. Testing to make it better – try to beat the control
  3. Testing big things – test things that improve marketing effectiveness
  4. Testing in isolation – multivariate testing is not practical
  5. Use statistically valid sample sizes – don’t base decisions on flawed methodology
  6. Testing what you can afford – test campaign variations that you can roll out
  7. Roll successes out slowly – confirmation testing leads to conclusive results

The most important thing? Don’t wing it; use campaign testing for better marketing.

Campaign Testing FAQs

What is campaign testing?

Campaign testing evaluates the effectiveness of marketing campaigns to improve brand strategy and achieve business goals.

Why is testing marketing campaigns important?

Testing marketing campaigns is important because it helps businesses identify what works and what doesn’t, enabling them to make data-driven decisions to improve their marketing efforts.

What are some things to consider when testing campaign effectiveness?

Seven key things to consider when testing campaign effectiveness include testing for the right reasons, testing to improve it, testing big things, testing in isolation, using statistically valid sample sizes, testing what you can afford, and rolling out successes slowly.

How do you determine the sample size for campaign testing?

To determine the sample size for campaign testing, businesses should work backward from their desired result to determine the initial audience size needed for statistically significant results. They can use tools like Optimizely’s A/B Sample Size Calculator or consult with experts for assistance.

What is the importance of confirmation testing?

Confirmation testing is important because it helps businesses ensure that the results obtained from initial testing are accurate and reflective of reality. Businesses can use data-driven decisions to roll out successful campaign variations by conducting confirmation tests.

A version of this post appeared in Red8 Interactive’s blog: 6 THINGS TO CONSIDER WHEN TESTING YOUR WEBSITE. Red8 is the team behind Inn8ly.

Author: James Hipkin

Since 2010, James Hipkin has built his clients’ businesses with digital marketing. Today, James is passionate about websites and helping the rest of us understand online marketing. His customers value his jargon-free, common-sense approach. “James explains the ins and outs of digital marketing in ways that make sense.”

Use this link to book a meeting time with James.