Some respondents answer survey questions with the same answer over and over again. They agree to everything, or they give identical numeric ratings to all items. It’s called “straight-lining,” and we often consider this a sign of poor data quality. I was intrigued by an approach to mitigating the problem I learned last week, shared…
A simple technique that will measurably improve your data quality is to offer survey response options that go from low to high, or from bad to good. Start with things like terrible, zero, poor, disagree, and so on. Then move towards spectacular, excellent, ten, and agree.
Yesterday we were in Milwaukee at a film studio, shooting for a soon-to-be-released Versta Research video about making research matter. It explains how to design and execute research to ensure that it gets heard, understood, and put into action by your business partners. The video outlines three tips. Here’s the first: begin by writing dream…
Difficult or lengthy surveys require hefty incentives, which unfortunately makes it tempting for some respondents to lie. We are facing this predicament in a survey right now. The survey requires two people from the same household who share household finances to complete parallel surveys. We are recruiting from an online research panel. Person 1 gets…
In our experience, the people who write the best questionnaires are researchers who spend their time immersed in complex survey methods and quantitative analysis but who focused on qualitative research earlier in their careers. Why? Because they are deeply sensitive to the dual objectives of survey design, namely: (1) Elicit data in specific ways and…
Earlier this year we trumpeted the idea of corporate research groups helping internal business partners do their own research. Sometimes is makes tons of sense, and sometimes not. So when it comes time for something more complicated, how do you educate a client about what matters and what doesn’t in terms of quality and rigor?…
There are plenty of good reasons to use Net Promoter Scores (NPS). They are based on sound measurement, are simple to use, are conceptually compelling, and are widely known by business executives outside of market research. But you know something is amiss when your bank starts asking how likely you are to recommend the online…
A perennial problem with open-ended survey questions is that respondents are lazy. It takes time and effort to think of good answers. And if the survey is self-administered online or on paper, it takes time and effort to write them out. But there are several things a savvy researcher can do. One of them is…
As more and more people use smartphones and mobile devices to fill out surveys, basic survey design decisions need to be revisited again and again. How should surveys be laid out visually? How should answer scales be constructed? How should people be recruited to take surveys? Some of the latest answers to these questions come…
Getting good responses to open-ended questions via online surveys is a challenge. So much so, that new technologies have evolved to probe on responses just as live interviewers used to probe people on phone surveys. But can you get really outstanding open-end responses to online surveys without fancy technology? The answer is yes, it just…