Why Your Neuro-Marketing Might Be 70% Wrong
Who can resist those flashy brain image scans showing that certain words, colors, images, or brands (or maybe your latest marketing message) magically light up consumers’ brains, presumably making them want to buy? Or, if you’re a cynic, who can resist sharing with colleagues the brain image scan of a dead salmon showing brain activity when presented with images of people in social situations?
Well, time to start resisting. Because most of them are probably wrong. A new research study (July 2016) just published in The Proceedings of the National Academy of Sciences suggests that the statistical software typically used by these brain imaging devices is flawed. Seriously flawed.
A team of scientists from Sweden and the UK in the fields of statistics, biomedical engineering, machine learning, and medical imaging science conducted a large empirical analysis of open-source resting-state fMRI data from 499 healthy controls. Since all of the scans were based on resting-state control subjects, they reasoned that the null hypothesis of “no effect” should prevail 95% of the time. But it didn’t:
In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of a number of fMRI studies and may have a large impact on the interpretation of weakly significant neuroimaging results.
They add later in the article:
It is important to stress that we have focused on inferences corrected for multiple comparisons in each group analysis, yet some 40% of a sample of 241 recent fMRI papers did not report correcting for multiple comparisons, meaning that many group results in the fMRI literature suffer even worse false-positive rates than found here.
What should you do? Here are three suggestions:
- If your neuro-marketing research effort is a thing of the past and it failed, take heart in knowing why. You were a brave pioneer. Don’t give up. Try again.
- If your neuro-marketing campaign is a thing of the past and it succeeded, congratulate your marketing team for their insights. Your fMRI data would have said they were geniuses no matter what. But in this case, they were!
- Most importantly, if you are in the middle of a neuro-marketing research or marketing campaign effort right now, get your data now. Have it re-analyzed. Insist that your vendor address the problems outlined here and come back to you with validated results. You should see the number of “statistically significant” data points drop dramatically. Recalibrate your strategy accordingly.
We still think neuro-marketing holds promise in helping understand how and why people respond to certain messages over others. But surely it will take some time for the neuro-marketing research industry to recode, rejigger, and revalidate its algorithms. Plus, it will take a lot more proof this time around (to this cautious brain, at least) that all those flashy images tell more than big fish stories.
OTHER ARTICLES ON THIS TOPIC:
