How to Interpret a Post-Hoc Test for ANOVA

Post-Hoc tests give you a closer look at the data.

An ANalysis Of VAriance, or “ANOVA” is a statistical procedure used to determine whether the differences between the means of three or more groups of data are statistically significant. One of the disadvantages of a one-way ANOVA is that it cannot tell you specifically which groups are different -- it can only tell you that somewhere between your groups, a difference exists. To get additional information, you need Post-Hoc tests.

Instructions

1. Run your ANOVA in your statistical analysis package, and select the “Post-Hoc” option, which may also be labelled “Follow up comparisons” or “Test for main effects.”

2. Decide whether you will adjust your significance level, and select the method of doing so. If you have only have three groups and are fairly confident -- due to previous tests or proven theory -- that your result will come out in a certain way, use “LSD.” Otherwise, select “Bonferroni” or “Tukey” if your package does not offer Bonferroni correction.

3. Wait for the results to appear on-screen, and then check that the overall “F” ratio of the ANOVA is significant. Scroll down to the table containing the “F” ratio, usually simply labelled “ANOVA,” and find the “Between groups” row. Check the significance on this row, usually labelled “Sig” or “p.” If it is above your alpha level (usually 0.05 -- consult your teacher or research partners to be sure), and you selected LSD in the previous step, stop here -- you have no significant differences. Otherwise, continue to the next step.

4. Scoll down to the table detailing your Post-Hoc tests, usually labelled “Multiple Comparisons.” Each group is listed down the first column, and its comparison with the other groups is given in the second column. Look for the “Sig” or “p” column, and scan down it to see if any results are lower than your alpha level. If the are, follow the row to the left to see which two groups this refers to. The means of these groups are statistically significant. Continue for the whole table.

5. Check to see how large the differences are. Sometimes statistically significant differences can be found when the actual differences between the means are very small. Visit uccs.edu/~faculty/lbecker/ and for the two groups you are comparing, enter their mean and standard deviation from your ANOVA results into the text boxes on the right of the screen. Click “Compute,” and then check the Cohen's d box. Effect sizes that are considered small, medium and large are .2, .5, and .8 respectively.

References

• University of Dayton: Using SPSS for One Way Analysis of Variance
• University of Colorado Colorado Springs: Calculate d and r using means and standard deviations
• SportSci.org: A Scale of Magnitudes for Effect Statistics