Direct Mail A/B Testing: How to Optimise Every Postcard Campaign
Most brands send one postcard design, measure the result, and draw conclusions that may be entirely wrong. Without a control, you cannot know whether your 4% response rate is good or bad, or whether a different design would have achieved 7%. Systematic A/B testing is what separates brands that improve their direct mail ROI over time from those that plateau. This guide explains how to design valid postcard tests, what variables to test first, and how to interpret results correctly.
What makes direct mail A/B testing different from email
Email A/B tests can reach statistical significance in hours with a list of thousands. Direct mail tests are slower and more expensive — each card costs real money to print and post, and response data can take 3–6 weeks to accumulate as customers act at different points in the redemption window. This means you need to be deliberate: test one variable at a time, ensure your sample size is large enough before drawing conclusions, and resist the temptation to call a winner too early. A minimum of 200 cards per variant is a practical floor; 500 per variant gives you more reliable data. For low-volume campaigns, you may need to pool results across multiple send cycles before reaching significance.
The four variables worth testing in order of priority
Offer first: the discount level, type (percentage off, fixed amount, free shipping, free gift), and expiry window have the largest impact on response rate and are worth testing before anything else. A 15% discount versus a €10 credit can produce dramatically different results depending on your AOV and customer psychology. Copy second: the headline and call to action drive whether a customer acts or discards the card. Test urgency framing ("Offer expires 31 May") versus benefit framing ("Your exclusive reward is waiting"). Design third: image-led versus text-led layouts, colour palette, and the prominence of the QR code all affect visual attention and response. Timing fourth: test sending 60 days post-purchase versus 90 days, or Monday delivery versus Thursday delivery. Timing tests require you to split your trigger conditions and are harder to set up cleanly — tackle them once you have optimised offer and copy.
Setting up a clean split test
A valid A/B test requires random assignment, not self-selection. Split your target segment alphabetically by customer ID or surname initial — odd versus even IDs, or A–M versus N–Z surnames. Do not split by geography or recency, as these introduce confounding variables. Use a unique promo code or QR URL per variant so you can attribute responses accurately. Run both variants simultaneously (or within the same week) to control for seasonal effects. Define your primary metric before sending — response rate (cards sent ÷ redemptions) is the cleanest metric. Revenue per card sent is more useful for business impact but requires a longer measurement window.
Interpreting results and avoiding common mistakes
The most common mistake in direct mail A/B testing is calling a winner too early. Allow a full 45-day redemption window before comparing variants — some customers take weeks to act on a postcard, and cutting the window short understates the slower-converting variant. Use a chi-squared test or an online A/B significance calculator to confirm that your difference is statistically meaningful and not random noise. A result is only actionable when p < 0.05. If your winning variant achieves a 2% higher response rate but the difference is not statistically significant, do not change your default design — run a larger test instead.
Building a testing roadmap
Structure your A/B testing as a quarterly programme rather than ad-hoc experiments. Q1: test your core offer structure. Q2: test your headline and CTA copy with the winning offer. Q3: test design and layout with the winning offer and copy. Q4: test timing and frequency with your optimised card. By year end, you will have systematically improved every major variable. Document every test, including those that showed no significant difference — negative results are valuable and prevent you from re-running the same test with the same inconclusive result.
Run your first A/B test with Mayday — create two postcard variants and split your audience in minutes.
Get started free