AP Finally Moves to Online Polling
The Associated Press (AP) announced last month it is moving all of its public opinion polling from phone surveys to online surveys. This is progress, because over the last decade AP has been one of a handful of media holdouts (along with ABC News and The New York Times) who have insisted that only phone-based polling is rigorous enough to meet its standards for publication.
Unfortunately it is not enough progress. The AP’s announcement, with its vague references to “scientific principles” seems to suggest there is only one type of online poll it will now accept, namely, polls from so-called “probability based” online panels. Quoting the AP:
Polling that draws on what are known as online panels is widely used for many purposes, but most of it does not meet AP’s standards for publication. That’s because typical online research leaves out the 15 percent of Americans who have no access to the Internet, and most online panels are made up of people who have clicked through online ads or been referred by friends to sign up to complete surveys in exchange for rewards.
Some of these concerns are valid, but beyond the issue of Internet coverage, the AP’s online polling will be subject to the same shortcomings of nearly every other online poll, namely:
- Respondents are opt-in volunteers who participate in surveys in exchange for rewards
- Low response rates mean that respondents self-select into certain surveys and not others
- Low response rates mean that the probability of inclusion in a survey is unknown
Indeed, one of the founders of the online panel now being used by AP noted in this month’s Journal of Survey Statistics and Methodology (JSSM) that its panel “currently reports a rather depressing response rate of approximately 2.5 percent.”
In reality, that’s OK, and the AP is on solid ground with its online polling. Why? Because there is plenty of data and research showing that many kinds of online polling work. Adding to that evidence, the Director of the Division of Behavioral Surveillance at the CDC cited preliminary findings just this month from a CDC pilot study showing “there is little practical difference between the results [that opt-in online panels] give and those from comparable probability-based surveys. In the results of our pilots thus far, I can find little to argue against their use, although we haven’t finished a full evaluation using health outcome and risk factor variables.” (JSSM, Volume 1, Issue 2, November 2013)
Versta Research applauds the AP for now accepting that online polling can work, and for moving it’s own polling to an online panel. But we also encourage them to align their standards with a broader, emerging consensus among pollsters that many types of online panels can provide valid and reliable estimates of public opinion and behavior.
