Practical Statistics for Data Scientists
Copy doc
3. Statistical experiments
A/B testing
An experiment with two groups to establish which is superior
IR
Intelligence Refinery

Why have a control group?

In a
properly designed A/B test:
Subjects are
randomized
to either treatments
Need to collect data on treatments A and B in such a way that any observed difference between A and B must be due to either:
Random chance in assignment of subjects
A true difference between A and B
Must establish a single test statistic before conducting the test
, otherwise leave room for researcher bias



image.gif
Why just A/B? Why not C, D…?

If testing which is the best out of multiple possible conditions, use
multi-arm bandit 
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.