Whenever I write a research report, I feel strongly ambivalent about flagging data as “statistically significant.” If possible, I try to avoid it altogether. Why? Because p-values and concepts of statistical significance are often misunderstood, misused, and misleading. Indeed, the problem is so prevalent that one well-respected scientific journal in social psychology (Basic and Applied…
A perennial debate among survey researchers is whether respondents should be allowed to answer “don’t know” or “not sure.” In fact, this is something we at Versta Research wrestle with (and argue about) internally almost every day. My approach is generally to not allow them. We have seen strong evidence in our own work that…
Actually, Joe Citizen does not love them, but Joe Researcher does. The horse-race polls for media release offer little value to the first Joe because the only poll that really matters is the election itself. But for Joe Researcher, political polls offer unparalleled, ongoing opportunities to understand, validate, and improve our methods of survey research.…
Some respondents answer survey questions with the same answer over and over again. They agree to everything, or they give identical numeric ratings to all items. It’s called “straight-lining,” and we often consider this a sign of poor data quality. I was intrigued by an approach to mitigating the problem I learned last week, shared…
Once a year we schlepp out the Google Analytics (yes, with good reason, only once a year—perhaps a topic for a future article) to find out what you, dear readers, are paying attention to. Traffic is higher than ever. During the last twelve months there were 383 visits per week to our website. But that…
The evidence continues to mount that phone surveys should be a method of last resort. Pew Research Center has released results of an experiment conducted in 2014 to test mode differences in how surveys are administered. The research question was whether people answer survey questions differently when responding to a real person on a telephone…
A simple technique that will measurably improve your data quality is to offer survey response options that go from low to high, or from bad to good. Start with things like terrible, zero, poor, disagree, and so on. Then move towards spectacular, excellent, ten, and agree.
Yesterday we were in Milwaukee at a film studio, shooting for a soon-to-be-released Versta Research video about making research matter. It explains how to design and execute research to ensure that it gets heard, understood, and put into action by your business partners. The video outlines three tips. Here’s the first: begin by writing dream…
Just one month after publishing our Quick Puzzle for Market Research Brains (SPOILER ALERT! If you have not tried the puzzle yourself, do it now before continuing to read!) which demonstrated the dangers of confirmation bias, I read a report from the National Science Foundation (NSF), released in May, that warned of the same thing.…
Leave it to the National Science Foundation to keep me hip on current research lingo. A report published last May (with the very unhip title Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science) offered this definition of an edgy new term in research called p-hacking: Generating many different sets of results and…