What is A/B Testing?

Definition

A/B testing compares two versions of an asset against the same audience to determine which performs better against a defined metric. Sound A/B tests require sufficient sample size, a single variable, and statistical significance.

Why it matters

Understanding A/B Testing is foundational to working in experimentation. It shapes how teams plan, communicate with stakeholders, and interpret performance.

Best practices

• → Define it in shared documentation so every team uses the same meaning.
• → Tie it to a measurable outcome rather than treating it as an end in itself.
• → Revisit assumptions quarterly — the meaning and benchmarks evolve.

Common mistakes

• → Treating it as a vanity metric instead of a decision input.
• → Copying competitors' targets without context for your funnel.
• → Optimizing it in isolation, hurting downstream conversion.