Subject-line tests are rounding errors: cold email campaign ideas that test the offer
Subject-line tests are rounding errors. The test that moves revenue is the offer.
On July 24 I got on a call with a partner whose calendars had filled faster than he could handle once his mailboxes scaled. His words: "we basically booked out all of july in like a week." Good problem. Then came the real question. He wanted new funnels beyond "want to be on a podcast?" He wanted campaigns for founders, campaigns for people building important things, angles that weren't the generic ask. And he had A/B testing switched on in his sending tool, and hadn't run a single test.
Booked out July in a week, then what?
That's the moment most people get campaign testing wrong. Once something works, the instinct is to protect it: keep the winning email, nudge a word here, try a different subject line, see if open rates tick up a point.
That's polishing. It's not testing. A one-point lift on a subject line is noise on most list sizes. What you want to know after a campaign works is whether a different offer, to a different slice of people, would work better. That's a question about the idea, not the wording.
I'd seen the size of that gap two days earlier on our own dashboard. On July 22 our sales and investor pitch campaigns were running around 0.43% interested leads per email, and our podcast invitation campaigns were at 1.69% and 3.3%. Same mailboxes, same lists, same team. The offer was the variable, and it was worth four to eight times the results. No subject line does that.
Angle tests: founders, builders, the invite
So my advice to him was to A/B test angles directly in the sending tool he already had. Not "Quick question" versus "Thoughts?" in the subject line. Whole different reasons to reply.
- A founders angle: an invitation built around the founder's story and what they've survived.
- A builders angle: an invitation for people working on something important, framed around the work, not the company.
- The plain invite, as the control.
Each of those attracts a different person and starts a different conversation. When one wins, you've learned something about your market, not just your copy.
Name tests: let replies decide
One of the cleanest offer tests I saw in July wasn't about the email at all. On July 6 a franchise client couldn't decide between two names for their podcast. Instead of debating it in a meeting, they planned to A/B test the two names in the invitation and let open and reply data decide.
That's the right instinct. The name of the show is part of the offer. "Come on a show called X" lands differently from "come on a show called Y," and your buyers will tell you which one they'd rather be associated with. You don't have to guess.
Another client on July 20 took a similar approach on their first sales campaign: an A/B test with AI picking the winner automatically, with podcast invitation campaigns to be layered in on top. Start with the offer you have, let the machine pick, then add the invitation and compare.
AI as the idea engine
The bottleneck in angle testing isn't the tool. It's ideas. Most teams run out of genuinely different angles after three or four.
So I stopped coming up with them myself. On July 6 my setup had AI proposing five campaign ideas a day, built from what had worked in past campaigns, with a one-click push to launch the ones I liked. By July 24 I'd moved it to ten new campaign ideas every day through the API, with the approved ones uploaded automatically.
Ten a day sounds like a lot. Most I throw away. But the two or three a week that make it through are angles I wouldn't have thought of, and they come with the history of what worked attached. The human job becomes judgment, not invention.
One variable at a time
The discipline that makes any of this readable:
- Change one thing per test. If you change the offer and the subject line and the audience, a win tells you nothing.
- Test the biggest variable first. Offer, then audience, then structure, then words.
- Give it enough volume. On a small list, wait for a few hundred sends per variant before you call it.
- Keep a log. Every test, the result, the date. The log is what feeds the next round of AI ideas.
- Kill losers fast and move the volume to the winner.
And be honest about what counts as a win. Interested replies, not opens. Opens are unreliable and tell you about the subject line at best. A test decided on opens is a subject-line test wearing an offer test's clothes.
Campaign ideas worth testing
If you need a starting list, these are offers, not wordings, and each is a real test:
- Pitch versus invitation to the same list.
- Two show names in the same invitation.
- Founder-story angle versus work-focused angle.
- An invitation to your main show versus an invitation to a pitch or panel format.
- A direct sales offer to one half, a podcast invite to the other, with the same follow-up.
Every one of those can move your rate by multiples. A subject line test might move it by a decimal.
If you want campaigns like these written, tested and swapped out for you, the sending engine is on how we work.
Frequently asked questions
What should I A/B test in cold email?
The offer first, then the audience, then structure, then words. In our campaigns, switching from a pitch to an invitation moved the lead rate four to eight times; subject lines move it by decimals.
How many variants should a cold email campaign have?
Two or three, changing one thing at a time. More than that splits the volume too thin to read on most lists.
How long should an A/B test run?
Until each variant has a few hundred sends, then pick the winner and move the volume to it. Logging every result makes the next round of ideas better.