How to Create a Direct Marketing Split Test

One of the hallmarks of direct marketing is the ability to test, measure and judge the success of the marketing effort by the metrics. By creating a split test, your direct marketing can be measured on the return on investment of your small business marketing strategy. A split test is the simplest head-to-head test between either the control marketing piece and a test piece or between what you think will work best and a test concept in the event that this is your first time marketing the item, business or concept. Here's how to create, execute and measure a split test in direct marketing.

Difficulty: Moderate

Instructions

Things You'll Need:

• Two direct marketing pieces. One piece is the control. The other piece is the one you test against the control.
• Unique URL, telephone number or other response vehicle to track results.
• Microsoft Excel to track the data.

1. Determine which media you will use for your direct marketing. Print ads and postcard marketing are two examples. Direct mail letters, television commercials, print ads and any kind of postcard marketing are common small business marketing efforts that can be tested.

2. Use your current marketing piece as the control. You should already know the response rate in percent and dollars, along with the costs for the control. If you do not have a control, your first direct marketing effort should be sent or distributed without a test to establish a control. You can also use your best guess as the control package, but this will not yield the most accurate results.

3. Using your control package or ad, make one change to it. You can only make one change. Making more than one change adds too many variables and you will not be able to tell which variable led to either the increase or decrease in response rate.

4. Common variables to test include the creative (copy or graphic design), the offer (low price, buy one get one, buy one get half off), or an incentive (free shipping vs. gift).

5. Be sure you have two separate response mechanisms in place, one for EACH creative - the control and the test piece. You need to be able to track orders and response back to the original creative piece. You can make up a unique URL and use URLs to track responses, a coupon code for each response, or a separate toll free number. As long as you can see the results from each piece, the control and the test panel, you are okay.

6. Mail, print or otherwise release the creative. Allow 6-8 weeks for the test and carefully track all data, including (depending on the medium): number of responses, number of new customers, number of return customers, average dollar order, etc.

7. Add this data to the Excel spreadsheet. Create a spreadsheet with separate columns for each data point. Make sure you track on separate worksheets or side-by-side the data from each campaign.

8. Now compare. Which piece gave you the higher response rate? The higher average dollar order? More new customers? That's the winner of your split test.

Tips & Warnings

• If creating unique URLs for your split test, make them short and easy to remember. Too many characters to type into a browser leads to mistakes and lost orders.
• Never change more than one variable: you won't be able to tell which variable improve response rates in your direct marketing if you change too much at once.

Resources

• Direct: Making Marketing Pay, biography of Lester Wunderman (father of direct marketing) - describes split tests in detail.
• Seven Oaks Consulting, direct marketing firm.

[JeanneGrunert]

on 11/9/2009 Thanks for your comments everyone!

James, there is a great free statistical program on www.drakedirect.com - website of Drake Direct consultants. Perry & Rhonda Drake were my direct marketing professors in the masters degree program at New York University. They offer a free program that can help you set up statistical cells for dm tests.

[askjamesholmes]

on 10/21/2009 Jeanne - This is an excellent step by step process for conducting a split test. I have found that by investing a little bit more time and conducting these test even the novice marketer can learn from and approve upon that which is being measured. I here this practice being taught more often and it is a best practice to be followed consistently.

I have just posted a post to my blog concerning a web based program that allows you to easily set up and track unlimited split test leveraging automation.

Thank you for your excellent contribution!

James

[chava812]

on 3/16/2009 Wonderful article on how to test what works best! Another favorite for me!!

[georgelarson]

on 1/30/2009 Good information on split marketing your direct mail marketing campaign. Thank you.

[kskaggs]

on 1/29/2009 Thanks!