Laughing at the Null Hypothesis
I’m not sure if this cartoon is funny because I don’t really get it. But it was interesting enough that I started thinking about the null hypothesis, and what it means, and how statistics is changing so quickly that “the null hypothesis” we learned about in our classes may soon feel like an odd artifact of history.
In statistics, the null hypothesis is like our legal maxim “innocent until proven guilty.” We look at data from a sample, and wonder whether a specific pattern we see in the sample reflects the full population it is supposed to represent. The answer is always NO (the null hypothesis) unless that pattern is really strong—that is, unless the evidence is overwhelming. How strong? The convention is that we have to be at least 95% certain the pattern we see could not be a chance event of random sampling.
It sounds simple (sort of) but it’s extremely convoluted, because we are not really calculating the probability that the null hypothesis (or conversely, our alternative hypothesis) is true or false. We are actually calculating the probability that our data could have “happened” assuming the null hypothesis is true.
Along comes Bayesian statistics, which is starting to turn this approach on its head. Now, instead of looking at the probability that our evidence happened by chance, we look at the probability that our hypothesis (null or otherwise) is true or false given the data at hand.
Yikes, this is already getting complicated. Bayesian statistics gets hard for two reasons. First, after so many years of doing statistics and thinking “backwards” in terms of data and hypotheses, Bayesian approaches are actually more intuitive, which gets our thinking completely mixed up. Second, practical use of Bayesian statistics often requires zillions more computations, so it can be challenging to “see” and feel comfortable with all the math behind it.
Fear not. We are pulling together a short primer on the underpinnings of Bayesian statistics for market research to be featured in an upcoming Versta Research newsletter. In the meantime, it’s worth casually mentioning at cocktail parties that you saw a new study that conclusively disproves the null hypothesis. Tell me if you get a laugh.
