Telephones used to be an awesome way to find a random sample of Americans. Nearly all Americans had a home telephone, nearly everybody would answer if you called, and most were willing to answer survey questions to help researchers understand public opinion. In fact, telephones have been so central to scientific survey research that the…
Over the last few years it has become clear that random digit dial telephone surveys—once the gold standard of survey research—are no longer so golden. But the New York Times has been a holdout among media giants, refusing to use online polls or to report findings from them. They have clung to the false idea…
One of the most useful techniques for in-depth qualitative interviewing is to ask respondents not only what they think, but what their friends, family, or colleagues think. It opens a whole new level of conversation and insight because all of the sudden respondents feel free to talk about their understanding and assessment of the world…
It’s been four years since the AMA convened a panel of thought leaders in market research to ponder questions about how social media fits into research practices and methods. An excerpt of our discussion was published in Marketing News in 2010. Last month, another top industry group (AAPOR) released its Emerging Technologies Task Force report…
Readers of this newsletter and blog know how much we value and respect the New York Times. It is one of the few resources we cite regularly, as it always has relevant, authoritative information that informs our work and that of our clients. So my heart sank when I read this description of a new…
Leading practitioners of survey research now agree that online polling and other forms of non-probability sampling can work. The key is to give careful attention to the survey protocols, data collection, cleaning, and back-end weighting so that ultimately the sample being analyzed matches the population you want to know about in as many relevant ways…
Question order in surveys matters a lot more than most of us realize, which makes it critical to assess what you most want to know before your survey respondents are influenced (or, in the language or survey experts, “primed”) by additional questions on the topic. Here is a superb example from last fall’s political polling…
We ran this script of a Christmas Eve phone survey interview two years ago. It’s time to bring it out again! It’s an amusing and oh-too-true parody of how market research pollsters ask questions in clear, precise, linear ways to gather data we need for analysis. Although it takes the form of a phone interview…
The Associated Press (AP) announced last month it is moving all of its public opinion polling from phone surveys to online surveys. This is progress, because over the last decade AP has been one of a handful of media holdouts (along with ABC News and The New York Times) who have insisted that only phone-based…
The purists in the polling industry who have always insisted that only probability sampling is valid may finally be accepting that their methods are dead. This is one happy conclusion I draw from the debate over AAPOR’s recent task force report on non-probability sampling, which is nicely summarized (and debated) as the lead article of…