The problem (so we’ve been told) with check-all-that-apply survey questions is that respondents do not process the full list of response options carefully and thoughtfully. Instead, they glance through, knowing they’re supposed to find at least a few that are relevant to them. They check the ones that jump out (often at the top of…
When the U.S. Census Bureau started redesigning its race and ethnicity question for the 2020 decennial census, we started revising our own short version for surveys, too. Now we have enough data from one full years-worth of surveying on multiple projects, and we can tell you this: it works really well. We used to have…
Last week we showed you a survey that might convince you to think twice about hiring a super high-end consulting firm for your survey research and insights. Fancy business consultants may know a lot about business, but they know little about gathering good data to answer their business questions. So this week, instead of focusing…
Ever wonder what you get if you hire a fancy schmancy consulting firm for your survey research and marketing insights, instead of Versta Research? Every once in a while we have a potential client who insists: “Give us IBM! Nobody ever got fired for buying IBM.” I got a view into what those surveys look…
Believe it or not, some people do not find puppies irresistibly adorable, as our newest honorary member of the Versta Research team (pictured here) can tell you. Elio arrived a couple of weeks ago. He is 13 weeks old and truly over-the-top adorable. He mostly sleeps in a crate next to me as I work…
What if a survey asks you a question that implies a yes-no answer (“Have you ever …”) but then offers you answer options that are different from what it implied (“often, sometimes, rarely, never”)? New research tells us that it depends on how the survey is administered. If the survey is online or on paper,…
The online customer satisfaction survey I received today had me noticing an unfortunate side-effect of the NPS (Net Promoter Score) schema and visual imagery. For those of you unfamiliar with NPS as a way to measure and track customer satisfaction, here is how it goes. In your survey ask: “How likely are to recommend Acme…
If you make your survey questions hard, it turns out that people will make a good faith effort to answer them. But they will make mistakes, and you will end up with bad data that lacks reliability and validity. What I find so interesting is that most people will actually try to answer your hard…
I once heard from an overzealous boss with a Ph.D. in political science that measurement scales should always have an odd number of scale points – the reason being that measurement scales should always have a neutral middle. We sometimes hear this from clients reviewing survey drafts as well: “Shouldn’t every question have a neither,…
If you want the most valid and reliable measures you can get, forget about NPS and other 11-point scales that so many companies use as their primary measure of customer satisfaction these days. It turns out that 2-point scales (yes vs. no, satisfied vs. dissatisfied, etc.) are far more reliable measures for gauging consumer attitudes.…