The site Which Test Won (www.whichtestwon.com), founded by Anne Holland, is a must-read for both IT and marketing professionals. Every week the site reports on an A/B test of a single page of an e-commerce to see which version of the page generate more click-throughs and/or orders.

Their tests seem narrowly focused on technology, but in reality they reflect everything we know or think we know about reaching and winning over customers. They provide the rarest of all marketing commodities -- real, verified, useful data about consumer behavior.

This week, WhichTestWon.com's Natalie Tomasso announced the creation of a guide to produce a 4-part presentation to convince the decision makers in your organization to approve and fund testing. The topic is website testing, but the principles are the same as for any marketing research. I'm sure you'll have no trouble translating it for your particular situation. I recommend that you download it.

Some version of their model should be used by every arts organization with every campaign, especially those which budget for multiple mailings. I'm often asked questions like, "Which image will work better?" and "Which promotion will generate more sales?" and I do my best to reason it out with my clients based on my knowledge and experience with arts marketing, and their knowledge and experience of their offerings and audience. But it's all guesswork. There are no hard and fast rules that work for every organization every time, so the only way to know for sure is to test.

It doesn't have to be hard or expensive, but the trick as always is with tracking and analyzing results. For example, for a few hundred dollars, you can split your print run in half and feather your mailing list into two equal parts so that you can test one image or slogan or promotion against another. The problem is with figuring out which sales came from which brochure. There are three common ways:

1) Separate ordering methods. Once upon a time, all orders came in by phone, fax or mail, so all you needed were codes on the order form and specific box office phone numbers for each version of the mailer. The rise of internet ordering makes things much more complicated. (And it's where the methods of WhichTestWon really come in handy.)

2) Order codes. Your box office can be trained and your online order form can be programmed to request the order code from the mailer. There is a lot of leakage -- some patrons give incorrect information or no information -- so it often helps to provide an incentive such as a discount or a freebie.

3) Address matching. This is the most time-consuming, but the most accurate method. Each order is compared against the test mailing lists to see which brochure the patron received. Some organizations try the short cut of segmenting mailing lists by zip code, which is OK if that's all you can do, but the results will be skewed by differences in demographics and geography. The best results come from feathering -- taking every other record, or every third record if you are testing three options.

Let me end by underscoring the importance of testing. Every market is different, and the optimal appeal for your situation is unique. Experts like me blah blah blah with the best of them, but we're just giving educated guesses. For best results, gather real data. Best of all, most research can be treated cumulatively. That is, what you learn from a test this season, can be applied next season while you test something else.

Happy testing!