Glossary · Analytics engineering and analytics

A/B testing

A controlled experiment in which users or cases are randomly assigned to two or more versions of something, such as a web page, message or process, and the outcomes are compared. Randomisation lets differences be attributed to the change rather than to other factors.

Why it matters

A/B testing gives stronger evidence of cause and effect than before-and-after comparisons, which are easily confused by seasonality and other changes. It is widely used to evaluate product features, pricing presentation and model-driven interventions.

Reliable tests need a primary metric chosen in advance, enough sample size, a fixed duration and guardrail metrics that catch harm, such as increased complaints or returns.

In practice

For example, a UK online insurer might test a redesigned quote page against the current version, with conversion as the primary metric and cancellations within the cooling-off period as a guardrail.

Where Rodan fits

Rodan designs experiments and measurement for digital products and AI interventions in Analytics and Intelligence work.

Related terms