Sum of Square Errors (SSE)

Calculate the overall mean of each group of data sets. For example, say there are two groups of data, set A and set B, where set A contains the numbers 1, 2 and 3 and set B contains the numbers 4, 5 and 6. The mean of set A is 2 ( found by adding 1, 2 and 3 together and dividing by 3) and the mean of set B is 5 (found by adding 4, 5 and 6 together and dividing by 3).

Subtract the mean of the data from the individual data points and square the ensuing value. For example, in the data set A, subtracting 1 by the mean of 2 gives a value of -1. Squaring this number (that is to say, multiplying it by itself) gives 1. Repeating this process for the rest of the data from set A gives 0, and 1, and for set B, the numbers are 1, 0 and 1 as well.

Sum up all the squared values. From the previous example, summing up all the squared numbers produces the number 4.

Calculating the Root MSE in ANOVA

Find the degrees of freedom for error by subtracting the total number of data points by the degrees of freedom for treatment (the number of data sets). In our example, there are six total data points and two different data sets, which gives 4 as the degrees of freedom for error.

Divide the sum of squares error by the degrees of freedom for error. Continuing the example, dividing 4 by 4 gives 1. This is the mean square error (MSE).

Take the square root of the MSE. Concluding the example, the square root of 1 is 1. Therefore, the root MSE for ANOVA is 1 in this example.