You might think that writing a feature article about liars, cheaters, and trolls in online surveys means we have a pretty damning view of online polls as a method of research. Not so. We are all in when it comes to online polling, and in fact it comprises the majority of the survey research we…
Sure we laugh at those ridiculously biased fundraising “surveys” from politicians. But for some reason, the poorly (but more cleverly) designed surveys put out by corporate PR departments often get a pass. Even worse, they sometimes get tons of visibility in both mainstream press and social media. I saw one such survey a few weeks…
If you grew up in the United States, you probably think big numbers are better when it comes to rating things. Higher scores on school exams are better. Higher scores in games and sports are usually better. Higher credit scores are better. Five-star restaurants are definitely better than one-star restaurants. In Germany it is often…
Bad surveys bug me. Bad surveys touted in The New York Times bug me even more, as I expect only the best from them. Bad surveys on the front page of The New York Times with ridiculous and sensationalist findings bug me so much that they inspire blog posts. A front page article on July…
Two audience members at our LIMRA talk (see Public Studies: Advice for the PR Team and Advice for the Research Team) asked questions about biased research findings. Doesn’t starting with the research story and writing dream headlines lead you down a path of finding or confirming what your internal client wants to hear? I suppose…
There is a small academic niche in market research that relies heavily on extremely cheap crowdsourcing for data. But before getting up in arms about the reliability and validity of such data, you should know this: They used to rely heavily on students enrolled in their college classes for data. Remember having to work six…
Here are two ways you might ask a question to document multiple behaviors, purchases, interests, etc. They seem like they would be equivalent, but they are not. The first is called a multi-check format: The second is called a yes-no grid: If respondents were super careful, thoughtful, and unbiased in how they answer, the information…
How many young African American men support Donald Trump? Not many. But if one of them ends up randomly selected into your survey, and if you use statistical weighting to ensure the sample does not skew too old and too white, then you are in trouble. This is what’s happening to the LA Times’ election…
There is an important directional bias in our culture such that things above tend to be perceived as better than things below. Top is better than bottom. Up is better than down. Over is better than under. Whatever the reasons for this bias (and there are several compelling theories) recent research shows that it affects…
Nine out of ten Americans (89%) regularly use the Internet. So the old argument that online surveys will miss big chunks of the U.S. population is no longer valid. But an important question remains: Are the remaining one in ten (11%) who are beyond reach with online surveys substantially different from the others? And if…