While it is tempting to use advanced programming to “force” good answers and good behavior out of survey respondents, don’t do it. Let respondents make mistakes. Let them mess up, and then review who messes up. This is one of the best ways to assess data quality in a survey, and trust me you will…
If you make your survey questions hard, it turns out that people will make a good faith effort to answer them. But they will make mistakes, and you will end up with bad data that lacks reliability and validity. What I find so interesting is that most people will actually try to answer your hard…
Thanksgiving is a time to reflect on all things that make our lives rich and rewarding, and my friend and colleague Henry (the farmer for whom I work on Saturdays) got me thinking about what I am thankful for in my work specifically, and in a new way. He asked us to think about who…
There is a “user-friendly” feature that some survey platforms now offer, and almost certainly you should avoid it. It is called auto-advance. I had an unpleasant encounter with it in a customer survey I just took. The idea is that, after clicking on an answer, instead of having to move a mouse or finger all…
It amazes me that so many marketing research surveys are not responsive and optimized for mobile devices. I understand why they are not optimized, as decent mobile surveys are hard to design, and no, all the junk platforms that tout mobile optimization are not good options. But if you’re hiring a research or consulting firm,…
Even just a few years ago the idea of using SMS text messaging for surveys seemed absurd. There were strong demographic skews to cell phone usage, and among SMS usage as well. Texting was often a cost add-on for cell phone users, so they carefully guarded it. And of course research companies could barely manage…
That Nespresso survey I got via my work email—surely it must be SPAM, right? I have no relationship with the company and have never inquired about it. Some sleazy market research company harvested a huge list of e-mails and blasted it out to hundreds of thousands of people like me. So I decided to report…
If you are a consumer-facing company deploying online surveys of your customers, accessible surveys will likely be on your agenda in the next couple of years. Accessible surveys incorporate unique design features so that people with disabilities who use assistive technologies can participate as fully as others. With the need for survey accessibility growing and…
Now that survey tools are easy to build and cheap to buy, the companies that sell these tools try to differentiate their offerings with various types of design enhancements. Sometimes they succeed by offering truly elegant respondent experiences with simple, intuitive, easy-to-use, and adaptable formats. Sometimes, though, it feels like all the razzle dazzle starts…
Some respondents answer survey questions with the same answer over and over again. They agree to everything, or they give identical numeric ratings to all items. It’s called “straight-lining,” and we often consider this a sign of poor data quality. I was intrigued by an approach to mitigating the problem I learned last week, shared…