Two audience members at our LIMRA talk (see Public Studies: Advice for the PR Team and Advice for the Research Team) asked questions about biased research findings. Doesn’t starting with the research story and writing dream headlines lead you down a path of finding or confirming what your internal client wants to hear? I suppose…
This familiar sentence opens the lead article of Public Opinion Quarterly’s 2017 special issue on survey research: “Telephone surveys are dead, claim some survey researchers, and should be replaced with nonprobability samples from Internet panels.” Hmm, I’ve said something similar to that in the past, I thought to myself. I guess I’m not the only…
There are no magic numbers for sample size. There is no such thing as a statistically significant sample. Unfortunately, those two words—statistically significant—are bandied about with such abandon that they are quickly losing their meaning. Even people who should know better (the data wonks at Google Surveys should know better, right?) are saying ridiculous things…
If there is one thing I hope you remember from your college statistics class, it’s this: Correlation does not imply causation. This is especially important to remember in our world of big data. Any large dataset will have hundreds of thousands of correlations, but most of those correlations will reflect purely random occurrences that mean…
At first I thought this was the dumbest survey question ever. Why ask people what device they are using to take the survey, when you know from the platform meta-data what device they are using? Yes, even lame self-service survey platforms can (and should be) capturing information about device, operating system, IP address, start time,…
It was so weird, right after the election, to see a media frenzy heaping praise and attention on the one poll that got it wrong. The USC/LA Times poll predicted a Trump win, with 47% of voters choosing Trump and 44% choosing Clinton. Right after the election, media headlines trumpeted its success: “How One Pollster…
Suppose your colleague left her job tomorrow and you needed to recreate, and double check, all of the work she produced for a research study just completed. You probably couldn’t do it. Think of every step along the way: exactly how the survey was programmed, how data were cleaned, why certain respondents and not others…
Data wonks have more fun than you may think. If you have not yet begun working with the R statistical program (which is mesmerizing, extremely powerful, hard to learn, but weirdly intuitive, and FREE) then here is a fun and relatively easy way to give it a test drive. It is becoming a tradition that,…
Sometimes data visualization is too compelling for it’s own good. People are impressed by the beautiful design, and they look right past the utter nonsense being presented. This is “beautifully” illustrated by Google’s rollout of the first of many new features of their survey tool. Here’s the email they sent: We’re excited to announce that…
Here’s a sobering question for professionals who work in the “insights” industry: How much of your market research really offers insight? Not just validating data, or routine measurements, or pro forma tracking. Not just percentages of “how many said what” in a customized study. By true insight we mean: a direct answer to a deeply…