The best way for you to get better survey data is to fix, optimize, and strategize your survey design. How? Abandon rote templates. Give up complexity. Think ahead to your analysis. Above all, put yourself in the shoes of a respondent. We market researchers complain when we get data back from surveys and see that…
This is the same survey I told you about a few weeks ago. It came back to me again, since I am part of the ABS-recruited probability-based panel that is fielding the survey. It is still one of the worst surveys I have ever seen. This time I took some screen shots: …
Survey questions that have multiple response options with an instruction to “select all that apply” are common in our industry, and for good reasons. They are efficient and easy for respondents to answer. They are efficient to lay out graphically. They shorten the length of a survey in terms of both time and space. The…
While it is tempting to use advanced programming to “force” good answers and good behavior out of survey respondents, don’t do it. Let respondents make mistakes. Let them mess up, and then review who messes up. This is one of the best ways to assess data quality in a survey, and trust me you will…
Having focused intently over the last several months on best practices for writing surveys (see Five Last Steps in Writing a Questionnaire and How to Write Surveys: 25 Best Practices) I have been paying extra attention to the work of research methodologists who codify best practices and develop protocols for implementing them. One set of…
Oddly enough, the researchers who most prize scientific rigor (our research colleagues in academia) are the last to have realized that online sampling can be treacherous. An article published in Wired magazine last week describes academic psychologists as “freaking out” because they just learned that a lot of their data, which they source from Amazon’s…
There is a “user-friendly” feature that some survey platforms now offer, and almost certainly you should avoid it. It is called auto-advance. I had an unpleasant encounter with it in a customer survey I just took. The idea is that, after clicking on an answer, instead of having to move a mouse or finger all…
If you worry that cell phones make it harder to get high quality, thoughtful, and richly detailed data with surveys, stop worrying. New research from a trio of survey measurement scientists (one at the Census Bureau and two at University of Michigan’s Survey Research Center) shows there are no differences in data quality and measurement…
This survey was done by a fancy business professor, in an endowed chair, at a top-ten university, along with a managing director at one of the world’s largest consulting firms. Then it was printed in one of our industry’s top publishing outlets. It takes less that a minute to scan this data and to realize…
Research firms like Versta (and research teams like yours?) tend to think of design, strategy, and analysis as the critical place where we add value. But guess what. Everybody thinks that, and one consequence is that fieldwork and data collection are becoming dangerously bad. Nobody is paying attention anymore. Fieldwork is on auto-pilot. We recommend…