It all depends on what counts as a “question,” and also on how complex those questions are. A grid question with multiple rows takes a longer to answer than a simple yes-or-no question. A question that allows you to select multiple items from a laundry list of options takes longer than a standard 4-point scale.…
The thing I love about strategists is that they know exactly what they want. By the time they come to us, they have analyzed their business needs. They have figured out what kinds of data will help them make decisions. They know why they want research, how they are going to use it, and how…
There is an important directional bias in our culture such that things above tend to be perceived as better than things below. Top is better than bottom. Up is better than down. Over is better than under. Whatever the reasons for this bias (and there are several compelling theories) recent research shows that it affects…
Now that survey tools are easy to build and cheap to buy, the companies that sell these tools try to differentiate their offerings with various types of design enhancements. Sometimes they succeed by offering truly elegant respondent experiences with simple, intuitive, easy-to-use, and adaptable formats. Sometimes, though, it feels like all the razzle dazzle starts…
Every year I read the university catalogues for summer coursework in research methods and share a few recommendations with colleagues and clients. If you are looking for more than just the information and inspiration you get from conferences, look to the substance and depth you can get from a classroom. It’s easier than you might…
A simple technique that will measurably improve your data quality is to offer survey response options that go from low to high, or from bad to good. Start with things like terrible, zero, poor, disagree, and so on. Then move towards spectacular, excellent, ten, and agree.
Difficult or lengthy surveys require hefty incentives, which unfortunately makes it tempting for some respondents to lie. We are facing this predicament in a survey right now. The survey requires two people from the same household who share household finances to complete parallel surveys. We are recruiting from an online research panel. Person 1 gets…
In our experience, the people who write the best questionnaires are researchers who spend their time immersed in complex survey methods and quantitative analysis but who focused on qualitative research earlier in their careers. Why? Because they are deeply sensitive to the dual objectives of survey design, namely: (1) Elicit data in specific ways and…
Earlier this year we trumpeted the idea of corporate research groups helping internal business partners do their own research. Sometimes is makes tons of sense, and sometimes not. So when it comes time for something more complicated, how do you educate a client about what matters and what doesn’t in terms of quality and rigor?…
If you ever have trouble separating all the hype of new methods and technology from the reality of whether those methods work, turn to the sober and decidedly non-hyped world of academic research. Some of the smartest people in the world build their careers by pursuing scientific research about research methods. They publish their results…