# A/B Testing in Cold Outreach: What to Test and When

Use an A/B test to check one business hypothesis with comparable prospect groups, not to choose the email your team happens to prefer. Test one variable at a time—such as the subject line, reason for reaching out, opening, or next step—and decide from meaningful replies and conversations rather than opens alone.

**Figures:** 5.93% — reply rate across our campaigns

## What an A/B test actually checks

Two completely different emails cannot produce a reliable conclusion. If the subject line, argument, tone and call to action all change, more replies in one version do not tell you what worked. A useful test answers one question: for example, whether production leaders respond better to a capacity-utilisation issue than to a delivery-time issue.

| Hypothesis | Version A | Version B |
|---|---|---|
| Subject line | A question about a current task | A short description of the reason for writing |
| Opening | An observation about the company | An observation about the sector |
| Next step | Offer to exchange details | Offer a short check of relevance |

Treat a subject-line test carefully. An open does not prove interest: a decision-maker may open an email and close it immediately. A stronger version produces clear replies such as a request for terms, relevant experience or a discussion after budget approval. See our guide to [B2B cold email subject lines](/en/guides/tema-pisma/) for the role a subject line can—and cannot—play.

One test means one changed element. Otherwise, a campaign collects impressions rather than evidence for the next decision.

## When to test: before scaling, not after a disputed result

Run a test when you have two credible versions of the same commercial hypothesis. An equipment supplier may believe that a commercial director cares most about less downtime, while a chief engineer cares most about predictable maintenance. These are separate hypotheses, with different recipients and reasons to start a conversation.

1. Write down the decision the test should confirm or reject.
2. Choose comparable company groups and one recipient role.
3. Keep everything unchanged except the element being tested.
4. Review the meaning of replies, not only their count.
5. Use the conclusion in the next wave or retire the hypothesis.

Do not test a message while also changing the segment, channel and offer. If a different audience receives a Telegram message instead of an email, you cannot credit the result to the new copy. For channel choices, first define the touchpoint route; see [multi-channel outreach for Russia and CIS](/en/services/multichannel-outreach/).

## How to recognise the genuinely stronger version

An email asking “Is this relevant?” can collect polite refusals and create an illusion of engagement. For a B2B team, a reply that moves the conversation forward matters more: a request for details, a question about terms, a referral to the right colleague, or agreement to a focused conversation. Compare willingness to continue the dialogue, not inbox noise.

One version promises to “optimise procurement”. Another names a concrete situation: the company is expanding its range and may be reviewing suppliers. The second email receives fewer formal replies, but several decision-makers ask about categories and working terms. The version with more short refusals is not the winner if the purpose is to begin a relevant commercial conversation.

- Does the report include reply texts and categories, not just totals?
- Are automated notices, refusals and commercial interest separated?
- Is it clear exactly what changed between the versions?
- Can the test conclusion be connected to the campaign’s next action?

If a lead reaches sales without the reason for contact and the reply thread, the test loses its value: you cannot see which hypotheses create substantive conversations. Read how to [route outreach replies into your CRM](/en/guides/peredacha-lidov-v-crm/).

## Mistakes that invalidate an outreach test

The most common mistake is declaring a winner after a random spike. Another is testing wording when the offer itself is unclear. If the recipient cannot see why a conversation would be useful, changing a few words in the subject line will not repair the proposition.

Mixing roles also distorts the result. A finance director and a production lead may hear about the same service, but they have different reasons to respond. A blended result does not show which argument lost; it shows that the audience was defined too broadly. Before testing, check your [ideal customer profile](/en/glossary/icp/) and segment logic.

- The test has one written question.
- The versions differ in one element.
- The groups are comparable by role and company type.
- The conclusion includes substantive reply analysis.
- The next campaign version is clear after the result.

## When A/B testing is not the right tool

A test cannot create market demand. If the product needs a long diagnostic process to explain, the commercial proposition is not yet clear, or the sale depends on a tender, one first email will not give an honest reading of potential. It can begin research, but it is not obliged to create a flow of meetings.

Testing is also a poor fit when there are too few target companies for comparable groups. With a narrow account list, review every response manually and refine the hypothesis; an [account-based marketing campaign](/en/services/account-based-marketing/) is better suited to that work. Do not test messages before contact quality is checked: a wrong address or irrelevant role will distort any conclusion, so start with [list verification](/en/services/list-verification/).

This is not for you if you need a guaranteed volume of meetings from an untested offer, or if your team cannot act on replies promptly. A test is useful when you are prepared to make a specific next decision from the evidence.

We define the hypothesis with your team, build comparable groups, maintain a reply summary and explain which decision the evidence supports. A strong version becomes the basis of the next wave; a weak one is taken out of circulation.

## Frequently asked questions

**What should I test first: the subject line or the email copy?**

Start with the riskiest assumption about why the recipient should have a conversation with you. If the reason itself creates no interest, a subject-line test will only produce a cosmetic conclusion. Once the reason is clear, test its presentation in the subject line and opening separately.

**Can I test email, Telegram and LinkedIn at the same time?**

You can compare channels as part of an overall outreach strategy, but that is not a clean copy test. Context, format and recipient expectations differ by channel. State the conclusion separately: which channel fits a particular role and task.

**Why can the version with more opens lose?**

A subject line can create curiosity without giving a business reason to reply. In B2B, questions, requests for details and willingness to discuss the task matter more. An open is an intermediate signal, not the campaign outcome.

**Should every new outreach campaign be tested?**

No. Test when you have a new audience, offer or disputed hypothesis. If a scenario is already proven for the same segment, preserve it rather than introducing uncertainty without a reason.

**What should an A/B test report include?**

It should state the hypothesis, the difference between versions, why the groups were comparable and how replies were classified. The essential part is the decision: what continues, what stops and why. This approach is part of a practical [outreach audit](/en/services/outreach-audit/).

---
Source: https://ot9.ru/en/guides/ab-testing-cold-outreach/ · OT9 (KAP Group) · updated 2026-08-10