How to Calculate Precision

Scientific calculator

Precision measures the repeatability of an event or sampling error. Take five shots at a basket and record the location each ball strike. If the ball hits near the same spot on the backboard, but misses the basket, its strikes are precise, just not accurate. Use standard deviation to measure precision.

Instructions

Determine Average, or Mean

1. Count the number, or N, of data values.

2. Add the data values together.

3. Divide the sum by N (from Step 1) to determine the average.

Determine Variance

1. Subtract the average from each data value to calculate each value's deviation.

2. Square each deviation.

3. Add the squares together.

4. Subtract one from the count, or N - 1.

5. Divide the sum of the squares by N - 1 to determine the variance.

Determine Precision

1. Take the square root of the variance to get the standard deviation.

2. Multiply the standard deviation by 100 to convert to percentage.

3. If the data values fall within one standard deviation below or above the average then they are considered precise values.

References

• University of Florida IFAS Extension: Determining Sample Size
• College of Saint Benedict, Saint John's University: Descriptive Statistics

Resources

• Read this Article in Spanish
• Chemistry Labs, University of Oregon: Uncertainty, Error, and Precision in Quantitative Measurements