Surveys run the gamut from silly to serious, and from sloppy to scientific. Occasionally we do something silly if a client insists (though we generally advise against it). But we never do sloppy surveys, and we hope you never do them either. Lean as far as possible towards rigor and science within reasonable constraints of…
Do you ever feel a little glow of patriotic pride in your research work? At Versta Research we sometimes do. Here’s why: the United States of America gives us and you—all free, and all easily accessible—one of the most amazing, useful resources available to every person and every business and every researcher: Census Data. It…
Surveys don’t get much simpler than this. It is from the New York Times, which I typically admire for its rigorous reporting, thoughtful analysis, and forward-thinking experimentation with data visualization. But even with such simplicity, this survey (published on April 22, 2018) manages to be terrible in at least three big ways: It asks a…
How many people do you think are college-educated in the United States? If you’re like most clients we work for (whose family members, friends and co-workers nearly all have college degrees) you will substantially over-estimate that number. The answer is a mere 29%. If you include associate’s degrees, then it goes up to 37%. These…
Bad surveys bug me. Bad surveys touted in The New York Times bug me even more, as I expect only the best from them. Bad surveys on the front page of The New York Times with ridiculous and sensationalist findings bug me so much that they inspire blog posts. A front page article on July…
This familiar sentence opens the lead article of Public Opinion Quarterly’s 2017 special issue on survey research: “Telephone surveys are dead, claim some survey researchers, and should be replaced with nonprobability samples from Internet panels.” Hmm, I’ve said something similar to that in the past, I thought to myself. I guess I’m not the only…
There is a small academic niche in market research that relies heavily on extremely cheap crowdsourcing for data. But before getting up in arms about the reliability and validity of such data, you should know this: They used to rely heavily on students enrolled in their college classes for data. Remember having to work six…
There are no magic numbers for sample size. There is no such thing as a statistically significant sample. Unfortunately, those two words—statistically significant—are bandied about with such abandon that they are quickly losing their meaning. Even people who should know better (the data wonks at Google Surveys should know better, right?) are saying ridiculous things…
It was so weird, right after the election, to see a media frenzy heaping praise and attention on the one poll that got it wrong. The USC/LA Times poll predicted a Trump win, with 47% of voters choosing Trump and 44% choosing Clinton. Right after the election, media headlines trumpeted its success: “How One Pollster…
Whenever you see those “margin of error” statements on political polls and other published surveys (even in market research industry publications, unfortunately!) take them with a grain of salt. They reflect only one type of potential error in surveys: namely, sampling error associated with sample size. Chances are that the “real” margin of error (if…