Statistical Significance
A determination that an observed relationship or difference in data is unlikely to be due to random chance.
What is Statistical Significance?
Statistical significance indicates that the results of a study (such as a difference between groups or a correlation) are unlikely to have occurred given the null hypothesis is true. It is typically assessed using a p-value compared against a pre-defined threshold (alpha), usually 0.05.
Why Statistical Significance Matters
It helps researchers separate signal from noise, providing a mathematical basis for deciding whether an effect observed in a sample likely exists in the broader population.
How to Interpret
Statistical significance does NOT mean practical significance. A result can be statistically significant but have an effect size so small that it is meaningless in the real world.
Example
If a study finds that a new teaching method improves scores by 10 points with p = 0.02, the result is statistically significant at the 5% level, suggesting the teaching method's effect is real, not just a random fluctuation in that specific sample.
Common Mistakes
- Equating statistical significance with practical importance.
- Assuming p > 0.05 proves that there is no effect (it only means there is insufficient evidence to prove an effect).