A/B testing in marketing

An A/B test compares two versions of an element (headline, banner, form) on a pre-chosen metric using cleanly split traffic. It replaces subjective “which looks better” debates with evidence and reduces the risk of hurting conversion after site or email changes.

Three columns—what A/B tests measure, why one hypothesis at a time matters, and how to run to significance—with arrows from clean splits to a metric decision.

An encyclopedia of digital marketing terms. See also all articles in «Marketer's terminology».

What it is

Variants A and B receive separate audience fractions with stable allocation; sample size, run length, and isolating unrelated changes all matter. Web tests need solid analytics; email tests must avoid mashing segments with different contexts into one readout without controls.

Why it matters for business

Teams get a repeatable decision process; stakeholders see KPI impact, not designer opinions alone. Compounding wins often lift CTR and conversion sustainably.

How to apply it

One crisp hypothesis per experiment. Run to statistical significance; don’t stop early on a lucky weekday unless planned. Tie tests to a CRO backlog and log outcomes to avoid repeating mistakes.

Table 1. Definition and business context

Criterion	In short
Definition	Variants A and B receive separate audience fractions with stable allocation; sample size, run length, and isolating unrelated changes all matter. Web tests need solid analytics; email tests must avoid mashing segments with different contexts into one readout without controls.
Why businesses care	Teams get a repeatable decision process; stakeholders see KPI impact, not designer opinions alone. Compounding wins often lift CTR and conversion sustainably.
Effect when done right	Stacked wins improve channel efficiency without only buying more traffic.

Table 2. Practice, ecosystem, and related terms

Area	What to consider
How to apply	One crisp hypothesis per experiment. Run to statistical significance; don’t stop early on a lucky weekday unless planned.
Works with	Anchors on CRO and analytics; maturity often extends into controlled personalization.
In the glossary	CRO: conversion rate optimization, Creative testing in advertising, CTR (click-through rate) for ads and snippets, Marketing personalization

Benefits and impact

Stacked wins improve channel efficiency without only buying more traffic.

How it fits the stack

Anchors on CRO and analytics; maturity often extends into controlled personalization.