The One Poll That Got It Wrong
It was so weird, right after the election, to see a media frenzy heaping praise and attention on the one poll that got it wrong. The USC/LA Times poll predicted a Trump win, with 47% of voters choosing Trump and 44% choosing Clinton. Right after the election, media headlines trumpeted its success:
“How One Pollster Saw Trump’s Win Coming”
“The USC/LA Times Poll Saw What Other Surveys Missed”
“What Can Be Learned From Only National Poll That Consistently Showed Donald Trump Leading”
And just last week, Donald Trump praised the poll for doing a great job and presumably getting it right.
The problem is that the USC/LA Times is one of the few polls that got it wrong. Their prediction that Trump would win was a fluke. They showed Trump winning the popular vote +3% over Clinton when in fact he was -2%. They got it backwards, and it was a huge miss. Their “prediction” was an accident; it was correct only because of the weird way in which electoral college votes misaligned with the popular vote.
To their credit, the research folks at the USC who are responsible for the poll acknowledge all of this:
As the initial shock of the election results subsided, everyone remembered that we actually had shown Trump ahead. We were inundated with calls and constant requests for interviews. Everyone wanted us to explain “the demise of polling” and how we got it right. As we have said many times, we did not actually get it right.
And even more to their credit, they have done the hard work of analyzing their data to figure out what went wrong. The culprit? They pulled a random sample evenly across all U.S. zip codes. But given the uneven ways in which populations are concentrated into zip codes, their sample over-represented rural (Trump) voters at the expense of urban (Clinton) voters.
It seems that the researchers weighted their data in standard ways (gender, age, ethnicity, etc.) but did not weight by geography. In analyzing what went wrong, they learned that adding geographic weighting brings their results into alignment with other polls that correctly estimated the vote.
Lesson learned: Weight Your Data carefully. Hey, that sounds familiar. It was Lesson #2 in our Winter Newsletter on this very topic.
