Whenever I write a research report, I feel strongly ambivalent about flagging data as “statistically significant.” If possible, I try to avoid it altogether. Why? Because p-values and concepts of statistical significance are often misunderstood, misused, and misleading. Indeed, the problem is so prevalent that one well-respected scientific journal in social psychology (Basic and Applied…
Every professional in market research should know about and use the amazing, public, free data resources provided by the U.S. government—data that includes essential information about our demographics, communities, businesses, commerce, roads, health, education…the list goes on. Now, a small group of consultants, data scientists, and programmers are trying to make that easier with a…
Every year I read the university catalogues for summer coursework in research methods and share a few recommendations with colleagues and clients. If you are looking for more than just the information and inspiration you get from conferences, look to the substance and depth you can get from a classroom. It’s easier than you might…
There is nothing worse than presenting research findings to an audience predisposed to hate it. Maybe you’ve had that experience of being in a boardroom of hostile managers? They don’t like what you’re about to say, so they pick at every methodological decision, shifting focus away from the story that the data have to tell.…
Market research almost always relies (or should rely) on U.S. Census data. The data are rigorous, reliable, and fundamentally important to every business, not-for-profit, and government body in the United States. So if you get a package that looks like this, as a neighbor did last week, here is what you should do: Open it…
If you ever have trouble separating all the hype of new methods and technology from the reality of whether those methods work, turn to the sober and decidedly non-hyped world of academic research. Some of the smartest people in the world build their careers by pursuing scientific research about research methods. They publish their results…
In Turning Data into Stories we pointed out that numbers have no inherent meaning. There are no essentially big or small or significant numbers, for example—they only become big or small or significant within the context of specific research questions and a range of possible answers. The same is true for p-values by which we…
Here’s a useful way to think about statistical significance. When looking at your data, what’s the probability that it looks like something is there, when in fact nothing is there. Randomness in data (because of sampling) often causes illusions. So testing for significance is all about measuring whether the patterns we see in our data…
One critique sometimes heard about NPS scores is that they can’t be stat tested. That is not true. Though I know of no statistical packages that do it automatically, it is an easy calculation you can do on your own. Here’s the math: n = sample size a = % promoters b = % passives…
You would think that the chancellor of a prestigious school like M.I.T.—a university that prides itself on practical applications of mathematics—would know something about how probabilities work with survey sampling. Apparently not. Quoting a New York Times article about M.I.T.’s survey of students about sexual assault on campus: