Online Surveys Have Same Accuracy as Phone
A new study presented by two professors from Harvard University and the University of Massachusetts at Amherst was probably one of the liveliest and potentially disruptive presentations at least week’s annual meeting of the American Association of Public Opinion Research (AAPOR) in Phoenix.
Why? Because their research challenges the beliefs of many AAPOR-ites who disregard most online research as being theoretically indefensible since it is not based on probability sampling. The research presented was based on parallel surveys conducted last year, designed to allow careful comparison of three survey modes: a “gold standard” phone survey that included both landline and cell phone RDD sampling, an opt-in Internet panel survey that used careful weighting techniques to ensure a comparably representative sample of the U.S. population, and a traditional paper-based mail survey. The questionnaires for each mode were nearly identical and elicited data on variety of topics including several that could be validated against the most rigorous U.S. Census and NIH data.
Here is how the authors summarized their results:
Comparing the findings from the modes to each other and the validated benchmarks, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates. (download full paper)
Their research adds to a growing body of research showing that rigorously done online surveys can be as useful and effective as rigorously done phone surveys.
True, most online surveys are not rigorously executed (but they can be), and the practical successes of online surveys have outpaced our understanding of the statistical, theoretical, and social underpinnings of why they work. But it is both intellectually closed-minded and pragmatically foolish to dismiss all online surveys as “invalid” and “not newsworthy” as some of our colleagues in AAPOR do.
Our approach at Versta Research is rooted in our training and continuing engagement in academic research, but we are also rigorously focused on doing what works to get the urgent answers and compelling business stories that our clients need. Online survey research is one crucial piece of that.
—Joe Hopper, Ph.D.
