One important part of hypothesis testing is understanding how to determine if there is enough evidence to support your claim. There are two possible outcomes of hypothesis testing:
Note that when it comes to statistics, nothing is certain. Rejecting the null hypothesis does not “prove” your claim to be true, it simply provides evidence that suggests that it could be. Similarly, failing to reject the null does not “prove” that the null is true, the data simply just did not provide enough evidence to suggest the alternative is true.
Multiple Hypothesis Testing
If you're conducting multiple hypothesis tests within a single study, the alpha stated in the decision rule will not match the alpha chosen for the study. In other words, if .05 is chosen as the alpha for the study, and 5 tests will be conducted, then the decision rule will state that the null hypothesis will be rejected if the p-value is less than .01, since the significance for each test is .01 using the Bonferroni Correction introduced on the Alpha & Beta page.
