A/B testing

An experiment with two groups to establish which is superior

Why have a control group?

In a properly designed A/B test:

Subjects are randomized to either treatments

Need to collect data on treatments A and B in such a way that any observed difference between A and B must be due to either:

Random chance in assignment of subjects

A true difference between A and B

Must establish a single test statistic before conducting the test, otherwise leave room for researcher bias

⁠

Why just A/B? Why not C, D…?

If testing which is the best out of multiple possible conditions, use

multi-arm bandit ⁠

⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.