Only one A/B test in seven produces a clear winner. The rules that make a test worth running, where to point it so the win compounds, and the five-step method for finding your own sending frequency ceiling.
Cyril Nie
Author
Great email programs aren’t built on one big idea. They’re built on hundreds of small wins, stacked over months. Testing is how you find the wins. Here’s how to do it right.
Only about 1 test in 7 produces a clear winner. Six end in a tie. That’s normal — it means your current setup is already decent. Don’t quit. The wins you do find keep paying on every future send, and testing costs almost nothing.
Change one thing at a time. Test a short subject vs. a long one. Not a short-with-emoji vs. long-without — you’ll never know which change mattered.
Send both versions at the same time. Version A on Tuesday vs. version B on Wednesday tests the day, not your idea.
Use enough people. Small list (under 5,000)? Test on 20–30% of it. Big list (50,000+)? 10–20% is plenty.
Wait for real results. Open-rate tests settle in about 2 hours. Revenue tests need a full day.
Use a significance calculator. Don’t eyeball it. Below 95% confidence, you don’t have a winner — you have noise.
A campaign test improves one send. A test on your welcome series improves every new subscriber from now on. Point your testing at your automated flows first, then at things that appear on every email: your from-name, your button style, your template. A subject line test helps once; a from-name win helps forever.
Subject lines get all the attention, but each one only helps once. Better bets: your from-name, your button (the words, the color, where it sits), your template, and how long your email is — try 100 words against 400 with the same offer. Then two tests that keep designers humble. The ugly email test: your pretty email against a plain one with no images at all. Plain often wins with business readers. And adding preview text if you don’t use it, which lifts opens about 5% for 30 seconds of work. Test one thing at a time. Testing lots of things at once needs far more people than most lists have.
Most tools can now guess when each person usually reads email, and hold their copy until then instead of sending everyone’s at 10am. That is worth 5–15% more opens, and you set it up once. It does not help in three cases. When time matters (a sale ending in four hours has to land now). When your list is under about 1,000 people (too little data to spot a pattern). And for receipts and order confirmations, which should arrive right away.
If you are guessing, here is what tends to work. Emails that land between 4 and 6am often win, because they sit at the top of the inbox when people wake up. Tuesday and Thursday are usually best for business readers. Monday inboxes are full, and by Friday nobody is listening. Weekends can beat weekdays for fashion, food, and fun. But fix time zones first. 10am your time is 3am for someone else.
“How often should I send?” has an answer you can test. More is usually better, up to a point. Sending 9–16 emails a month returns $46 for every $1. Sending once a month returns $13. Open rates hold steady from monthly all the way up to twice a week. They only start to drop near daily. Here is how to find your own line:
Example numbers, not yours. Your limit is the email just before unsubscribes start climbing.
Record today’s numbers: revenue per email, unsubscribe rate, click rate.
Add one send a week for four weeks.
If each email makes less money but your total for the week goes up, the extra email is earning its place.
If unsubscribes climb more than 0.1 points per send, you have gone too far.
If clicks fall more than 10% across the board, pull back.
Better still, don’t pick one number for everyone. Use the tiers from Chapter 4: fans get everything, ghosts get almost nothing. Brands that do this keep their revenue and lose 20–40% fewer subscribers.
Write down what you tested, what you guessed would happen, and what actually happened. Ties count too — learning that button color doesn’t matter for your audience frees you to test something that does. Aim to test something in at least 20% of your sends.
Rarely, for single sends. Detecting a realistic difference in click rate needs thousands of recipients per variant, so most small-list tests measure noise. Testing inside automated flows, where results accumulate over months, is the workaround.
Things with large possible effects and lasting value: the offer, the format of the email, the trigger timing of an automation, the number of steps in a flow. Button colors and single-word subject line tweaks do not move enough to detect.
Raise cadence gradually and watch unsubscribe rate, complaint rate and revenue per email. Your ceiling is the point where revenue per email starts falling faster than total revenue rises. That number is specific to your audience, not to your industry.
Discover more insights on similar topics
The complete 2026 guide to email marketing: 16 chapters on list growth, segmentation, automation, deliverability, testing, attribution and AI. Built from 908 sources and 4,798 insights.
Email returns about $36 for every $1 spent — roughly 30 times the next best channel. Here is why it keeps winning, and the six parts every working program is built from.
Apple inflates open rates and Gmail’s scanners click your links. Here are the email metrics that survive scrutiny, the thresholds worth holding yourself to, and why your own last 90 days beat any industry average.