- What are two variables that you would be interested in studying?
- How would you articulate a null hypothesis and alternative hypothesis for your two variables of interest?
What you must understand about the relationship between probability and statistics is that the results of any statistical analysis are never 100% certain. It may be necessary for you to spend several hours on this topic to completely grasp the concepts and relationships among statistical significance, a null hypothesis, and a p-value. In the meantime, you may find the following guidelines useful:
1. The researcher starts by articulating a "null" hypothesis and an "alternative" hypothesis. In the context of a t-test, for example, the null hypothesis predicts "no difference." By contrast, the alternative hypothesis predicts a “significant difference.”
2. To test the null hypothesis, the researcher (you), sets the "alpha level" before conducting a hypothesis test. The alpha level determines when to reject a null hypothesis. In the social sciences, an alpha level of 0.05 (i.e., 1 in 20 chance) is almost always appropriate.
3. In contrast to the alpha level that is set prior to looking at data, the obtained value is the value that the researcher calculates using statistical software. The obtained value is often called the "p-value." It is the observed probability value, and it determines whether or not to reject the null hypothesis. In the context of a t-test, when the null hypothesis is rejected, there is “a statistically significant difference.”
4. For example, if the alpha level is .05, and the observed p-value ( p) for a t-test shown in the statistical software is less than .05 (e.g., p =.03, p =.002), the researcher would reject the null hypothesis of no difference.
5. Therefore, to determine statistical significance, follow these rules:
. If the observed p-value is GREATER THAN the alpha level of .05, do not reject the null hypothesis; there is not a statistically significant difference.
. If the observed p-value is LESS THAN the alpha level of .05., reject the null hypothesis; there is a statistically significant difference.
To learn more about the logic of hypothesis testing, read "Hypothesis Testing" in the Navarro et al. (2019).textbook:
· Navarro, D. J., Foxcroft, D. R., & Faulkenberry, T. J. (2019). Learning statistics with JASP: A tutorial for psychology students and other beginners. https://learnstatswithjasp.com/
. Chapter 8, “ Hypothesis Testing [PDF].”