What the Polls Show: Research Works!
I have never been a media poll watcher or politics junkie who tracks every new poll saying which candidate is favored to win. It is hard to see the point of spending so much time and money predicting the outcome of an event that will be known with certainty within a matter of days, weeks, or months. But elections are an amazing way to see survey research methods in action, and there are few opportunities to have those methods validated so quickly, accurately, and unforgivingly than political polling.
What did we learn from the polls this election season? We learned that even with the huge challenges confronting research these days—declining response rates, disappearing landline telephones, shifting modes of communication, and overwhelming volumes of data—research works. Surveys work, including both telephone surveys and online surveys. Sophisticated data aggregation and statistical model-building work. And all of these methods can all work amazingly well.
Indeed, I and my colleagues in the research industry were excited by the election regardless of who we hoped would win because the industry showed so convincingly that research methods work. Methods were pushed to the limit with phone screening techniques and data weighting, auto-dialing, and non-probability Internet sampling. Of course not every poll hit the mark, but taken together, we had a remarkably consistent view of what would happen—and it did.
I especially appreciated this commentary from Nate Silver, a statistics blogger with the New York Times. He found himself rebutting the research naysayers, and criticizing media outlets for trying to present “balanced” stories by saying the election was close, a claim inconsistent with the data. Four days before the election he wrote:
To argue that Mr. Romney is ahead, or that the election is a “tossup,” requires that you disbelieve the polls, or that you engage in some complicated interpretation of them….
Some argue that the polls are systematically biased against Republicans. This might qualify as a simple argument had it been true on a consistent basis historically, but it hasn’t been…. Others argue that undecided voters tend to break against the incumbent, in this case Mr. Obama. But this has also not really been true in recent elections…. A third argument is that Mr. Romney has the momentum in the polls…. This may be the worst of the arguments, in my view. It is contradicted by the evidence, simply put.
Silver became a voice for all of us who argue that careful, systematic, and rigorous analysis of data can speak volumes.
If you have colleagues who disbelieve research, or who diminish the value and reliability of what it can do, talk to them about this election. Show them that rigorous research is possible. Remind them that if they want to make smart and strategic decisions, they are better off with great research than they are with wishful thinking.
–Joe Hopper, Ph.D.
