Practical Statistics for Data Scientists
3. Statistical experiments

icon picker
A/B testing

An experiment with two groups to establish which is superior

Why have a control group?

In a properly designed A/B test:
Subjects are randomized to either treatments
Need to collect data on treatments A and B in such a way that any observed difference between A and B must be due to either:
Random chance in assignment of subjects
A true difference between A and B
Must establish a single test statistic before conducting the test, otherwise leave room for researcher bias

Why just A/B? Why not C, D…?

If testing which is the best out of multiple possible conditions, use
multi-arm bandit 
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
) instead.