A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix. Why? Because their research challenges the beliefs of many AAPOR-ites who disregard…
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone…
With the magic of the holidays upon us, we got to thinking about “magic” in market research, or the lack thereof. So our just- published quarterly newsletter focuses on magic numbers in market research, arguing that certain “magical” numbers seem to guide much of what we do, whether we realize it or not. The best…
The idea of sampling is not hard to grasp, but methodological discussions about sampling can quickly move into the higher reaches of mathematics and probability that confuse even researchers who are not statisticians. Ever wonder what we are talking about when we refer to “probability samples?” Or the rationale for not reporting margins of error? …
Many assume that rigorous market research always works with random samples or probability samples. This assumption is not true. There are many studies that require purposive sampling instead. Purposive sampling involves finding people with specific characteristics or qualities, even if they do not fully represent the whole population, because these specific people can provide unique…
Should you state a survey’s margin of error in your press release when pitching a story to the media? In our view, the answer is no. Why not? Because margins of error refer to sampling error only, not about the overall accuracy or error of the survey itself. But how many readers of your news…
The idea that online panel surveys can replace telephone surveys ruffles feathers among my colleagues at the American Association of Public Opinion Research (AAPOR). So what would they think of using Twitter posts as a substitute for phone surveys? The idea seems crazy, but as reported in Science, researchers at Carnegie Mellon University have found…
If something works and it keeps on working but you don’t know exactly why it works, what would you do? Our view is that you should keep doing it. Not everyone agrees with us. The American Association of Public Opinion Research (AAPOR) convened a task force to study online survey panels, and released their report…
Ten years ago, surveys through online panels were rare. Mostly we conducted telephone surveys. Today it is the other way around. With online survey panels being a $2 billion industry in the U.S., there is now a wealth of comparative data to measure and assess the implications of using online surveys as a substitute for…
Telephone surveys are still considered the gold standard for rigorous public opinion polling and market research. The reason is that virtually every household in the U.S. can be reached by telephone, and therefore we have careful methods of determining the probability that any individual person is included in a sample to be surveyed. Knowing this…