Getting Started
The primary purpose of this app is to help you visualize scenarios whose probability (i.e., the p-value) is either very large or very small. For the researcher engaged in hypothesis testing, it is the low probability scenarios that are the most exciting. These low probability scenarios look just like a real treatment effect (i.e., the Treatment and Control groups are clearly different) — and yet they can occur entirely due to random selection and random assignment. When a researcher sees a p-value of .05 or less, it indicates that the Treatment and Control groups are so clearly different, that random selection and random assignment would rarely result in such an outcome. To see why researchers prefer low probability outcomes when deciding whether to reject the null hypothesis, click on each of the probabilities below — the app will automatically configure the scenario.
Try each of the scenarios (by clicking the link below):
- (No treatment effect; the Treatment and Control group means are identical)
- (Large treatment effect; the Treatment and Control groups are clearly distinct)
After checking out both scenarios, two additional options will appear below — Type I Error and Type II Error. At any time, scroll down to the ‘Scenario Controls’, where you can move the p-value sliding bar (to see corresponding scenarios), select a treatment effect level (none, small, medium, or large), and adjust the sample size.
The treatment has no effect on the dependent variable.