One of the “best practices” for survey research I learned in graduate school and then early in my career is this: Survey measurement scales should have at least five points. Presumably that provides enough room for variation, and it provides a neutral midpoint as well. But it is not true. Or, like all best practices,…
If you have trouble getting people to fill out your surveys, you ought to be paying them. There is nothing wrong with it. It does not mess up your data with phony information that people were paid to provide. On the contrary, paying an appropriate incentive helps signal the importance of your survey such that…
There is an important directional bias in our culture such that things above tend to be perceived as better than things below. Top is better than bottom. Up is better than down. Over is better than under. Whatever the reasons for this bias (and there are several compelling theories) recent research shows that it affects…
A perennial debate among survey researchers is whether respondents should be allowed to answer “don’t know” or “not sure.” In fact, this is something we at Versta Research wrestle with (and argue about) internally almost every day. My approach is generally to not allow them. We have seen strong evidence in our own work that…
Some respondents answer survey questions with the same answer over and over again. They agree to everything, or they give identical numeric ratings to all items. It’s called “straight-lining,” and we often consider this a sign of poor data quality. I was intrigued by an approach to mitigating the problem I learned last week, shared…
A simple technique that will measurably improve your data quality is to offer survey response options that go from low to high, or from bad to good. Start with things like terrible, zero, poor, disagree, and so on. Then move towards spectacular, excellent, ten, and agree.
In our experience, the people who write the best questionnaires are researchers who spend their time immersed in complex survey methods and quantitative analysis but who focused on qualitative research earlier in their careers. Why? Because they are deeply sensitive to the dual objectives of survey design, namely: (1) Elicit data in specific ways and…
Earlier this year we trumpeted the idea of corporate research groups helping internal business partners do their own research. Sometimes is makes tons of sense, and sometimes not. So when it comes time for something more complicated, how do you educate a client about what matters and what doesn’t in terms of quality and rigor?…
Two things bugged me about the market research survey invitation I got from my alma mater. First, it was a mediocre three star survey that asked me a bunch of information they could easily have gotten from my LinkedIn profile. Second, it offered a chance to win a prize if I participated, and the effort…
A perennial problem with open-ended survey questions is that respondents are lazy. It takes time and effort to think of good answers. And if the survey is self-administered online or on paper, it takes time and effort to write them out. But there are several things a savvy researcher can do. One of them is…