While it is tempting to use advanced programming to “force” good answers and good behavior out of survey respondents, don’t do it. Let respondents make mistakes. Let them mess up, and then review who messes up. This is one of the best ways to assess data quality in a survey, and trust me you will…
If you make your survey questions hard, it turns out that people will make a good faith effort to answer them. But they will make mistakes, and you will end up with bad data that lacks reliability and validity. What I find so interesting is that most people will actually try to answer your hard…
Here is a sly little test I recommend giving the next researcher who offers up their help on your next quantitative survey. Ask if they have ever used a DIY survey tool, and what advice they would give you about sample size based on this selection tool you looked at from one of the largest…
Customizing MaxDiff exercises by piping in text from previous survey answers might seem like a good idea, but analyst beware: if you are estimating individual-level scores using hierarchical Bayes or latent class analysis, you must split the data into subsets before you calculate the scores. Your results will be nonsense otherwise. Here is the scenario…
Thanksgiving is a time to reflect on all things that make our lives rich and rewarding, and my friend and colleague Henry (the farmer for whom I work on Saturdays) got me thinking about what I am thankful for in my work specifically, and in a new way. He asked us to think about who…
Surveys run the gamut from silly to serious, and from sloppy to scientific. Occasionally we do something silly if a client insists (though we generally advise against it). But we never do sloppy surveys, and we hope you never do them either. Lean as far as possible towards rigor and science within reasonable constraints of…
Nobody gets really good at market research by taking classes. It comes from learning on the job and in the field from other researchers. That is because researchers are always solving new problems and answering new questions that have never been asked before. That inspired us to make our summer newsletter The Big “How To”…
If you want the most valid and reliable measures you can get, forget about NPS and other 11-point scales that so many companies use as their primary measure of customer satisfaction these days. It turns out that 2-point scales (yes vs. no, satisfied vs. dissatisfied, etc.) are far more reliable measures for gauging consumer attitudes.…
You might think that writing a feature article about liars, cheaters, and trolls in online surveys means we have a pretty damning view of online polls as a method of research. Not so. We are all in when it comes to online polling, and in fact it comprises the majority of the survey research we…
This week one of our sample suppliers said “no” when we asked if we could re-contact respondents for a purpose other than our original request. Good thing I asked, and you know what? I’m glad they said no. I want suppliers to think carefully about seemingly simple requests, and to point out ways in which…