Testing a survey before it launches takes about 4 to 6 hours of an experienced professional’s time, for a standard ten- to fifteen-minute survey. If you don’t invest that time up front, you are almost certain to discover mistakes while in the field (or worse: after), which can make for a nightmare of programming corrections…
Contrary to some widely held beliefs that mobile devices are a barrier to survey participation, a majority of survey respondents nowadays fill them out on phones, rather than on desktops, laptops, or tablets. Research has amply documented that the quality and reliability of data collected via mobile devices is comparable to data collected on desktops.…
It is possible now to get thousands of responses to a survey overnight so that you can turn around research results as quickly as your managers and clients want. Would you trust the findings? I hope not. Suppose you want a general population survey that truly represents all U.S. adults. You will need to ensure…
In basic stats class, all of us learned about the importance of random sampling. It provides the foundation for the iron-clad mathematics of estimation, statistical significance, and margins of error. But the problem in social science, market research, and opinion polling is that random sampling (sometimes referred to as probability sampling) almost never exists. So…
Telling stories is a natural human behavior, but also highly complex and difficult to imitate or automate. This means that, happily, trolls can’t tell stories. And asking survey respondents for an extremely brief story (even just a few words) about their experience related to a survey topic can help you differentiate real respondents from fake…
A survey platform we have been using launched a new service promising an “expert review” of our surveys before we send them out for data collection. Unfortunately, it is a gimmick. It turns out that there are no experts reviewing our surveys. An artificial intelligence robot reviews them. It was interesting to see which best…
Let’s suppose that customers want to be surveyed. What is the best way to do it? Should we call them, send email, use text messaging, push surveys through mobile apps, or send old-fashioned paper surveys through the mail? This is a question we rarely ask in market research, because for most survey research we desperately…
What if a survey asks you a question that implies a yes-no answer (“Have you ever …”) but then offers you answer options that are different from what it implied (“often, sometimes, rarely, never”)? New research tells us that it depends on how the survey is administered. If the survey is online or on paper,…
Any good researcher should agonize over mode effects in surveys. Mode effects are differences in statistical estimates caused by the “mode” through which respondents take a survey. If there are mode effects, then how the survey is conducted (by telephone, online, through a smartphone app, in person, or by mail) will affect the results, requiring…
Conducting surveys with your own customers is surprisingly difficult anymore. Phone options are dead, and online surveys are tough, because nobody answers or responds anymore. So what’s a researcher, strategist, or marketing professional to do? How about conducting a paper-based mail survey. It sounds crazy, but we just did it, and it succeeded beyond what…