Problems With Attitude Surveys

Attitudes or feelings are difficult to measure with a survey.

Although they can provide useful data, a badly written or administered survey designed to determine people's attitudes can give you faulty information. Even the best attitude surveys can result in ambiguous data because attitude is a very difficult concept to measure.

Confusing Questions

It is very easy to ask survey questions in such a way that the results will be inaccurate or skewed toward a particular point of view. If a survey includes questions that are worded in a confusing way, a significant variation exists in how participants interpret and answer them. For example, a confusing survey question might ask: "Should you not spank your children to punish them?" Let's say a respondent thought that spanking was unacceptable. Because the question is worded poorly, they could choose to respond in either the affirmative or negative and mean the same thing. They could be thinking, "No, you should not spank your children," or "Yes, you should not spank your children."

Leading Questions

From the time they are young, people are taught what sorts of attitudes are acceptable and unacceptable. For example, losing a job is not supposed to inspire happiness. If the survey writer is biased, they might sometimes use these attitude "rules" to get the response they want, even though it is not necessarily accurate. Here is an example of a leading question: "Senator X has overburdened our state economy, leading to the loss of thousands of jobs. How do you feel about his job performance?" Even if a respondent actually liked the politician they were being asked about, they would respond more negatively because of the leading details that were added to the question.

Socal Taboos

In some instances, the rules that govern appropriate attitudes are so strong that people might be unwilling to answer a survey honestly. Let's say that someone is taking a survey that measures their attitudes toward racism. They might be asked the question: "White and black children should all go to the same schools. How strongly do you agree or disagree with this statement?" Even if the respondent honestly believes in segregation, the stigma against that attitude is so great that he may choose to lie on the survey and give a more socially acceptable answer.

It's All Relative

Even in a well-constructed survey that doesn't deal with difficult topics, attitude can still be extremely difficult to measure because feelings are relative. Let's say you wanted to conduct a survey to measure the quality of life in a particular neighborhood and you ask the question: "How satisfied are you with the level of access to medical care that you have?" The problem with this is that every respondent will use her own method for evaluating their level of satisfaction. An elderly man with cancer might feel dissatisfied because he needs more care than any facility in the area can give him. On the other hand, a college student with no health problems might be very satisfied because they only need an annual checkup. As you can see, this question might not give you any objective insight into medical care access or quality of life.

References

• Instructional Assessment Resources: Writing Survey Questions