Practical Statistics for Data Scientists
Share
Explore
Practical Statistics for Data Scientists
3. Statistical experiments

icon picker
Hypothesis tests

Tests whether random chance might be responsible for an observed effect
IR
Intelligence Refinery

The null hypothesis

The baseline assumption that any difference between the groups is due to chance
One way to test the null hypothesis is via
resampling permutation procedure
⁠
⁠
Shuffle together the results from groups A and B, then repeatedly deal out the data in groups of similar sizes, then observe how often we get a difference as extreme as the observed difference
The nature of the null hypothesis determines the structure of the hypothesis test
Creates a “null” probability model and tests whether the observed effet is a reasonable outcome of that model
⁠

The alternative hypothesis

The opposite of the null hypothesis, where the difference between groups is due to the treatment
Taken together, the null and alternative hypotheses must account for all possibilities
⁠

One-way, two-way hypothesis test

One-way (one-tail) test
Extreme chance results in only one direction count toward the p-value
Suited for A/B decision making, as one option is assigned as "default” unless the other one proves to be better
Two-way (two-tail) test
Extreme chance results in either direction count toward the p-value
More conservative, many statisticians use it to avoid argument

The null hypothesis
The alternative hypothesis
One-way, two-way hypothesis test
Gallery
Share
 
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.