What Is the Effect of Small Sample Sizes on Data Evaluation?
Research scientists interpret data gathered during studies and use their interpretations to draw conclusions. It is important that the studies be conducted according to scientific guidelines; if studies are not conducted correctly, the data gathered will be inaccurate and lead to false conclusions. One important guideline is that the sample size must be large enough. If the sample size is too small, it is difficult to tell whether a particular result occurred due to coincidence rather than because of what scientists did.
Other People Are Reading
-
How Studies Work
-
In most studies, scientists manipulate one variable, or changing condition, in order to try to answer a question. They divide the people involved in the study into two groups. One group is called the control group, and the other group is called the experimental group. People are assigned randomly to each group so that there is no chance that the scientists' biases influence the results. The number of people involved in the study is known as the sample size.
How Results are Interpreted
-
After a study is completed, scientists list all the results without identifying which people reacted in particular ways. Sometimes the scientists will not label which results came from the control group. These safeguards are taken so that scientists will look honestly at the results and will not be influenced by their own ideas. Once data is compiled, scientists perform statistical calculations to determine how likely particular results are.
-
Sample Size and Results
-
If the sample size is small, the results look very different from the way they look if the sample size is large. For example, suppose scientists study how playing video games after school affects seventh graders' grades on their homework. If only 10 students are involved in the study, almost everyone in each group might have the same grades. If 1,000 students are involved in the study, grade distribution might be more diverse.
Statistical Significance
-
Scientists determine whether or not results of a study are meaningful by performing a calculation known as statistical significance. Statistical significance refers to the probability that results occurred by random chance. This probability must be less than .05 in order for the results to be statistically significant.
When working with small sample sizes, the probability of a chance result goes up. For example, if all five of the students who played video games after school each day scored lower on homework than the five who did not, this result is not statistically significant. If 670 out of 1,000 students score lower on the same homework given the same conditions, it is far less likely to be due to random chance.
Confounding Factors
-
In order for data to be interpreted accurately, scientists must design studies carefully so that the only variable that changes is the one studied. If more than one variable could contribute to a particular result, it becomes difficult to tell what the reason is for the result. Extra variables are called confounding factors.
When a sample size is small, the likelihood of accidentally increasing confounding factors goes up. This is due to the fact that larger groups of people are likely to be more diverse than small groups. For example, in a study of 10 students, many of the students are likely taking a class with the same teacher, have similar interest levels in the class or have the same basic understanding of the subject matter. These confounding factors could cause students to earn certain grades on a test regardless of whether or not they play video games after school.
-
References
- Photo Credit data 61 image by chrisharvey from Fotolia.com
