At first I thought this was the dumbest survey question ever. Why ask people what device they are using to take the survey, when you know from the platform meta-data what device they are using? Yes, even lame self-service survey platforms can (and should be) capturing information about device, operating system, IP address, start time,…
It amazes me that so many marketing research surveys are not responsive and optimized for mobile devices. I understand why they are not optimized, as decent mobile surveys are hard to design, and no, all the junk platforms that tout mobile optimization are not good options. But if you’re hiring a research or consulting firm,…
Almost certainly more than you think. If you are purchasing access to survey respondents from panel providers, or from survey software providers, or from any other source beyond your own list (and even then, how carefully are you controlling access to your survey?) you are probably getting fraudulent data from automated bots or from survey-taker…
The thing I love about strategists is that they know exactly what they want. By the time they come to us, they have analyzed their business needs. They have figured out what kinds of data will help them make decisions. They know why they want research, how they are going to use it, and how…
Using surveys to “quiz” people about their knowledge (versus asking them about their attitudes or behaviors) is tricky. First, online surveys make it easy for people to look up answers they may not otherwise know. Second, people want to get quiz answers right, and so they are motivated to cheat. This is the reason I…
A surprising trend among our academic colleagues in marketing, psychology, and the social and behavioral sciences, is that they are crowdsourcing respondents for research studies from Mechanical Turk (MTurk). I am still overcoming my own resistance to the idea, as it rubs against Versta Research’s obsessive focus on data quality. Online sampling absolutely can and…
Even just a few years ago the idea of using SMS text messaging for surveys seemed absurd. There were strong demographic skews to cell phone usage, and among SMS usage as well. Texting was often a cost add-on for cell phone users, so they carefully guarded it. And of course research companies could barely manage…
For all the high level intellectual work you do in research (statistical work, design work, analysis, or writing) there is just as much brute force effort you need to put into fieldwork. That means finding the people to interview, convincing them to participate, asking them questions, and recording their answers. Even if you outsource, keeping…
Some respondents answer survey questions with the same answer over and over again. They agree to everything, or they give identical numeric ratings to all items. It’s called “straight-lining,” and we often consider this a sign of poor data quality. I was intrigued by an approach to mitigating the problem I learned last week, shared…
A simple technique that will measurably improve your data quality is to offer survey response options that go from low to high, or from bad to good. Start with things like terrible, zero, poor, disagree, and so on. Then move towards spectacular, excellent, ten, and agree.