If you make your survey questions hard, it turns out that people will make a good faith effort to answer them. But they will make mistakes, and you will end up with bad data that lacks reliability and validity. What I find so interesting is that most people will actually try to answer your hard…
Surveys run the gamut from silly to serious, and from sloppy to scientific. Occasionally we do something silly if a client insists (though we generally advise against it). But we never do sloppy surveys, and we hope you never do them either. Lean as far as possible towards rigor and science within reasonable constraints of…
If you want the most valid and reliable measures you can get, forget about NPS and other 11-point scales that so many companies use as their primary measure of customer satisfaction these days. It turns out that 2-point scales (yes vs. no, satisfied vs. dissatisfied, etc.) are far more reliable measures for gauging consumer attitudes.…
You might think that writing a feature article about liars, cheaters, and trolls in online surveys means we have a pretty damning view of online polls as a method of research. Not so. We are all in when it comes to online polling, and in fact it comprises the majority of the survey research we…
Sure we laugh at those ridiculously biased fundraising “surveys” from politicians. But for some reason, the poorly (but more cleverly) designed surveys put out by corporate PR departments often get a pass. Even worse, they sometimes get tons of visibility in both mainstream press and social media. I saw one such survey a few weeks…
This survey was done by a fancy business professor, in an endowed chair, at a top-ten university, along with a managing director at one of the world’s largest consulting firms. Then it was printed in one of our industry’s top publishing outlets. It takes less that a minute to scan this data and to realize…
Here’s an unfortunate reality in the world of survey research: Survey questions can sometimes be copyrighted. When certain questions (no matter how obvious and common) are asked together as a set, and then scaled into a single measure (no matter how obvious and simple), and then validated in a study as being a solid tool…
At first I thought this was the dumbest survey question ever. Why ask people what device they are using to take the survey, when you know from the platform meta-data what device they are using? Yes, even lame self-service survey platforms can (and should be) capturing information about device, operating system, IP address, start time,…
The thing I love about strategists is that they know exactly what they want. By the time they come to us, they have analyzed their business needs. They have figured out what kinds of data will help them make decisions. They know why they want research, how they are going to use it, and how…
Using surveys to “quiz” people about their knowledge (versus asking them about their attitudes or behaviors) is tricky. First, online surveys make it easy for people to look up answers they may not otherwise know. Second, people want to get quiz answers right, and so they are motivated to cheat. This is the reason I…