Are They Cheating or Helping? New Research on Survey “Cheating” Raises Thorny Issue
An endearing trait of survey respondents is that most are sincerely interested in helping you. They want to offer authentic opinions. They want to provide you with good data. I was reminded of this when reading findings from a study about survey “cheating” in a recent issue of Public Opinion Quarterly.
The researchers (at the University of Mannheim, Universitat Pompeu Fabra, University of Göttingen, and University of Michigan) explored whether respondents look up answers to questions that are designed to measure political knowledge. They used tracking technology to assess whether people switched away from the survey, and also asked people whether they switched away to look up answers.
They found that one-third of their survey respondents, did, in fact, “cheat” by looking up answers (32% based on tracking data, and 33% based on self-report).
But take a look at how they asked the questions:
How many member states does the European Union consist of? Please, answer the question as accurately as possible.
Who is the current President of the European Commission? Please, answer the question as accurately as possible.
How many delegates does the European Parliament currently have? Please, answer the question as accurately as possible.
Did you search for the answers to any of the three previous questions on the European Union on the internet? It is very important for our research that you answer this question truthfully.
If you ask me, looking up the correct answers to these questions is not cheating. They are doing what they were instructed to do. They are doing everything they can to answer as accurately as possible!
This study actually shows how helpful respondents are. They try to do a good job. If they do not know the answer to your question, by gosh, give them a minute and they will look it up.
Of course there are plenty of fraudsters and people who will cheat to get into surveys. Finding them requires more rigor than even most researchers realize. But it is always important to understand the surprising ways in which good respondents may provide “bad” data because of something you did when writing your questions.
—Joe Hopper, Ph.D.