With the magic of the holidays upon us, we got to thinking about “magic” in market research, or the lack thereof. So our just- published quarterly newsletter focuses on magic numbers in market research, arguing that certain “magical” numbers seem to guide much of what we do, whether we realize it or not. The best…
The idea of sampling is not hard to grasp, but methodological discussions about sampling can quickly move into the higher reaches of mathematics and probability that confuse even researchers who are not statisticians. Ever wonder what we are talking about when we refer to “probability samples?” Or the rationale for not reporting margins of error? …
Many assume that rigorous market research always works with random samples or probability samples. This assumption is not true. There are many studies that require purposive sampling instead. Purposive sampling involves finding people with specific characteristics or qualities, even if they do not fully represent the whole population, because these specific people can provide unique…
Should you state a survey’s margin of error in your press release when pitching a story to the media? In our view, the answer is no. Why not? Because margins of error refer to sampling error only, not about the overall accuracy or error of the survey itself. But how many readers of your news…
Despite having been on the front lines of social research for twenty five years, interviewing respondents personally and eliciting data through surveys, I still feel somewhat surprised and disbelieving that people really want to participate in research. But they do. Sometimes eagerly. Almost always truthfully. Surely, my surprise stems from my own reluctance to fill…
The idea that online panel surveys can replace telephone surveys ruffles feathers among my colleagues at the American Association of Public Opinion Research (AAPOR). So what would they think of using Twitter posts as a substitute for phone surveys? The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found…
If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report…
Ten years ago, surveys through online panels were rare. Mostly we conducted telephone surveys. Today it is the other way around. With online survey panels being a $2 billion industry in the U.S., there is now a wealth of comparative data to measure and assess the implications of using online surveys as a substitute for…
Telephone surveys are still considered the gold standard for rigorous public opinion polling and market research. The reason is that virtually every household in the U.S. can be reached by telephone, and therefore we have careful methods of determining the probability that any individual person is included in a sample to be surveyed. Knowing this…
A research article the Winter 2009 issue of Public Opinion Quarterly provided some useful comparisons of data quality between phone surveys and various types of online surveys. The findings are based on an experiment that fielded identical questionnaires via three survey modes, and, not surprisingly, there are strengths and weaknesses to each type of survey.…