I recently came across this strange big book, the Reference Manual on Scientific Evidence, compiled by the Federal Judicial Center and the National Research Council. It’s not fun reading, but it provides an enormously useful and fascinating overview of what makes social and marketing research (and other kinds of scientific research, too) sufficiently rigorous to…
Just last week a client proposed having their salespeople recruit customers to take the loyalty and satisfaction survey we had launched for them. It’s a terrible idea, for exactly the reasons that Dagwood realizes with his baffled look in this cartoon. If you want honest feedback that your business unit can honestly use to improve,…
I’ve always felt suspicious of consultants who claim to spend the bulk of their time solving business problems and offering insights and, by the way, they also do market research. Then I get a look at their research, and my suspicions are confirmed. So often the data is a MESS—an absolute mess that baffles me…
The next time you analyze the results of your customer satisfaction or brand loyalty tracking study, and you notice an upward or downward shift, ask yourself this: Is it reasonable to think that certain customers—either the happy ones or the unhappy ones—were more willing to give you their opinions than the other group? If so,…
It’s the data. We want it all. And to be fair, this article is less about Qualtrics, and more about what makes for truly good and rigorous research. It just turns out that Qualtrics is the one tool that makes it more difficult and more expensive to do that kind of research than any others…
Telephones used to be an awesome way to find a random sample of Americans. Nearly all Americans had a home telephone, nearly everybody would answer if you called, and most were willing to answer survey questions to help researchers understand public opinion. In fact, telephones have been so central to scientific survey research that the…
Last week we wrote about social media and market research, and suggested that industry experts agree it is not yet shaking the foundations of how we do research. So what about the other gigantic Internet development of the last decade—mobile technology? It should be shaking the foundations of how we do research. Current estimates are…
It’s been four years since the AMA convened a panel of thought leaders in market research to ponder questions about how social media fits into research practices and methods. An excerpt of our discussion was published in Marketing News in 2010. Last month, another top industry group (AAPOR) released its Emerging Technologies Task Force report…
Leading practitioners of survey research now agree that online polling and other forms of non-probability sampling can work. The key is to give careful attention to the survey protocols, data collection, cleaning, and back-end weighting so that ultimately the sample being analyzed matches the population you want to know about in as many relevant ways…
Question order in surveys matters a lot more than most of us realize, which makes it critical to assess what you most want to know before your survey respondents are influenced (or, in the language or survey experts, “primed”) by additional questions on the topic. Here is a superb example from last fall’s political polling…