A vivid illustration of a statistically spurious relationship is this: People who wear hats tend to drive slower. But there is no causal relationship, as hats don’t cause people to drive slower. It’s just that older people wear hats more than younger people, and older people tend to drive slower. That’s actually kind of interesting,…
A persistent misperception in our industry is that technology can (or eventually will) largely automate our work. It reflects a fundamental confusion between the tools that we use versus the work that we do. This confusion was evident in a recent New York Times article about the surprising amount of time that data scientists devote…
I’ve always felt suspicious of consultants who claim to spend the bulk of their time solving business problems and offering insights and, by the way, they also do market research. Then I get a look at their research, and my suspicions are confirmed. So often the data is a MESS—an absolute mess that baffles me…
Before the days of DATA MINING this was called DATA DREDGING. And for good reasons, because it was considered not a good thing. Almost any dataset will have nonsense correlations, and if you’ve turned on your magical statistical testing tool at the 95% confidence level, five out of every one hundred nonsense correlations will likely…
Every two years marketing executives from more than 60 corporate members of the Marketing Science Institute (MSI) vote on the most important marketing topics they need to understand better. They represent companies like Kraft, Colgate-Palmolive, Bank of America, Nielsen, and Walmart. MSI—a think tank, of sorts, whose mission is to “bridge the academic and business…
Versta Research has just published is spring 2014 newsletter which features a short video on how to do pricing research. If you ever work with price and demand data, and need to figure out the optimal price for your product or service, here are two great tools worth knowing about: 1. PowerPoint (yes, that says…
I just read an online interview with Bob Muenchen, author of R for SAS and SPSS Users. He nicely describes an important aspect of R that I found to be one of the weirdest and most difficult to grasp, but once I “got it,” fell in love. Quoting at length:
There was a recent online discussion among colleagues who are active in the American Association for Public Opinion Research (AAPOR) about experiences with and recommendations for survey analysis and statistics software. Many noted that traditional commercial software is getting extremely expensive, but that newer options do not always have the same capabilities that many of…
We love this cartoon because it raises such interesting issues about probability and prediction in market research. Here’s a quiz. Suppose you own a retail store and want us to help predict which customers that come through the door will buy. You want to focus your sales efforts on the best prospects. We do our…
Over the last few years we have wondered whether spreadsheet software like Excel will soon make statistics software like SPSS or SAS obsolete. Spreadsheets have amazingly powerful and often intuitive capabilities. They have many of the statistical functions we use every day. Younger people entering our profession rarely know programs like SPSS or SAS, and…