Here is a sly little test I recommend giving the next researcher who offers up their help on your next quantitative survey. Ask if they have ever used a DIY survey tool, and what advice they would give you about sample size based on this selection tool you looked at from one of the largest…
If you conduct a well-designed, well-executed survey of 500 people you probably think (and yes, we may have told you) that your margin of error is ±4%. It’s beautiful. It’s all based on math. And if you ever had that “aha” moment in statistics class, you probably agree that the mathematical proof of it is…
Whenever you see those “margin of error” statements on political polls and other published surveys (even in market research industry publications, unfortunately!) take them with a grain of salt. They reflect only one type of potential error in surveys: namely, sampling error associated with sample size. Chances are that the “real” margin of error (if…
Here’s a useful way to think about statistical significance. When looking at your data, what’s the probability that it looks like something is there, when in fact nothing is there. Randomness in data (because of sampling) often causes illusions. So testing for significance is all about measuring whether the patterns we see in our data…
One critique sometimes heard about NPS scores is that they can’t be stat tested. That is not true. Though I know of no statistical packages that do it automatically, it is an easy calculation you can do on your own. Here’s the math: n = sample size a = % promoters b = % passives…
Readers of this newsletter and blog know how much we value and respect the New York Times. It is one of the few resources we cite regularly, as it always has relevant, authoritative information that informs our work and that of our clients. So my heart sank when I read this description of a new…
Every year I enjoy getting an invitation to participate in Bon Appétit’s “most important survey of the year” because it provides so many good lessons for survey research. This year it has me thinking about the absurd e-mail subject line that says: “We need your help to get 100% participation.” Bon Appétit has 1,442,388 subscribers. …
Some business managers and marketing executives mistakenly believe that “big data” will deliver better insight because of the sheer volume of data now at our disposal. Now we just need the statisticians, the computing power, and the analytics software to sift through it all, right? Not so. The truth is, for most purposes you don’t…
What IS a credibility interval, you ask? It is a term making its way into mainstream market research just as Bayesian statistics are making their way into market research. A credibility interval provides a range of values, calculated using Bayesian statistical techniques, within which a statistical estimate is likely to fall. It is analogous to…
Way back in 1944, Edwards Deming published an article in the American Sociological Review that could be required reading for anybody who does research today. He outlined all potential (and unfortunately, common) sources of error in survey research. Apparently our contemporary obsession with sample sizes, random samples, response rates, and margins of error is not…