Get Rid of Those Survey Speeders
If you found that 10% of your online survey respondents were ignoring your questions, generating random data, and there was an easy way to identify who those respondents were, what would you do? I hope you would delete them, go back to the field, and replace them. But for some reason there are those among us who think bad data is OK as long as it is randomly bad.
Recently published research in Public Opinion Quarterly looked at respondent speeding behavior in online surveys, measuring case-wise and page-specific speeding across nine different online surveys. Not surprisingly, they found that most speeders are giving random answers such that “speeding primarily adds some random noise to the data.” As such it does not alter marginal distributions, and although it may weaken correlations, it does not shift the nature of those correlations and any conclusions derived from them.
Of course this is exactly what random data would do, and surely it is good to know that the data from speeders is random. But unfortunately, the authors go one step further saying their findings “shed some doubt on whether it is worthwhile to identify and remove speeders.” Why? Because leaving them in does “not necessarily lead to wrong conclusions.”
Wait a minute. Everything is watered down, correlations become harder to see, but our marginal distributions stay the same, so who cares? Yikes. I care. And I desperately hope our clients care, too. I like to think our clients pay us for the most rigorous research we can do. They pay us to deliver the deepest insights we can get. They don’t pay us merely to avoid wrong conclusions.
In our view, watered-down, “mostly” right research is not good enough. Identifying speeders is not hard, and laziness is the only reason to keep them in. Do the right thing and get rid of those survey speeders.
