Whenever you see those “margin of error” statements on political polls and other published surveys (even in market research industry publications, unfortunately!) take them with a grain of salt. They reflect only one type of potential error in surveys: namely, sampling error associated with sample size. Chances are that the “real” margin of error (if…
How many young African American men support Donald Trump? Not many. But if one of them ends up randomly selected into your survey, and if you use statistical weighting to ensure the sample does not skew too old and too white, then you are in trouble. This is what’s happening to the LA Times’ election…
A surprising trend among our academic colleagues in marketing, psychology, and the social and behavioral sciences, is that they are crowdsourcing respondents for research studies from Mechanical Turk (MTurk). I am still overcoming my own resistance to the idea, as it rubs against Versta Research’s obsessive focus on data quality. Online sampling absolutely can and…
The last presidential election was a stunning win for online polls and a drubbing defeat for telephone polls. It provoked a loud and contentious debate among survey methodologists about the accuracy of polls and about how (and whether) to revise our methods. So here we are again, four years later with technology having moved faster…
Polling and phone technologies may finally have reached the point where mobile phones are no longer a threat to phone-based surveys, but a superior alternative. Only half (52%) of U.S. households now have a landline telephone. In contrast, more than nine out of ten adults have a mobile phone. And the latest findings from Pew…
Nine out of ten Americans (89%) regularly use the Internet. So the old argument that online surveys will miss big chunks of the U.S. population is no longer valid. But an important question remains: Are the remaining one in ten (11%) who are beyond reach with online surveys substantially different from the others? And if…
Actually, Joe Citizen does not love them, but Joe Researcher does. The horse-race polls for media release offer little value to the first Joe because the only poll that really matters is the election itself. But for Joe Researcher, political polls offer unparalleled, ongoing opportunities to understand, validate, and improve our methods of survey research.…
There are good reasons to avoid phone surveys if you can. But if there is no alternative, it is critical to think about strategies for integrating cell phones into your outreach and sampling. Almost half (48%) of U.S. adults have no landline telephone service, while fewer than one in ten (9%) have no cell phone…
Having jumped back into the 1,016-page Reference Manual on Scientific Evidence from the Federal Judicial Center and the National Research Council (see last week’s article on Defending Your Statistics in Court), I keep finding fascinating nuggets of information worth sharing with our marketing and research colleagues of all stripes. Consider this snippet from the section…
I resist the idea of using Twitter for research because I do not know anyone beyond journalists, PR people, and marketing folks who actually use Twitter. Well, that’s a terrible reason to dismiss a potentially rich source of data, especially because one premise of survey research is that small parts can represent the whole. My…