For the longest time I was puzzled by what it means to “over-fit” a model to one’s data. How can a model describe one’s data too perfectly? Aren’t we trying to get as close to a perfect fit as possible? This one-minute video will show you an example of over-fitting. After watching it, the odd…
Before the days of DATA MINING this was called DATA DREDGING. And for good reasons, because it was considered not a good thing. Almost any dataset will have nonsense correlations, and if you’ve turned on your magical statistical testing tool at the 95% confidence level, five out of every one hundred nonsense correlations will likely…
Every two years marketing executives from more than 60 corporate members of the Marketing Science Institute (MSI) vote on the most important marketing topics they need to understand better. They represent companies like Kraft, Colgate-Palmolive, Bank of America, Nielsen, and Walmart. MSI—a think tank, of sorts, whose mission is to “bridge the academic and business…
Leading practitioners of survey research now agree that online polling and other forms of non-probability sampling can work. The key is to give careful attention to the survey protocols, data collection, cleaning, and back-end weighting so that ultimately the sample being analyzed matches the population you want to know about in as many relevant ways…
Versta Research has just published is spring 2014 newsletter which features a short video on how to do pricing research. If you ever work with price and demand data, and need to figure out the optimal price for your product or service, here are two great tools worth knowing about: 1. PowerPoint (yes, that says…
I just read an online interview with Bob Muenchen, author of R for SAS and SPSS Users. He nicely describes an important aspect of R that I found to be one of the weirdest and most difficult to grasp, but once I “got it,” fell in love. Quoting at length:
Question order in surveys matters a lot more than most of us realize, which makes it critical to assess what you most want to know before your survey respondents are influenced (or, in the language or survey experts, “primed”) by additional questions on the topic. Here is a superb example from last fall’s political polling…
Every year I enjoy getting an invitation to participate in Bon Appétit’s “most important survey of the year” because it provides so many good lessons for survey research. This year it has me thinking about the absurd e-mail subject line that says: “We need your help to get 100% participation.” Bon Appétit has 1,442,388 subscribers. …
I’m not sure if this cartoon is funny because I don’t really get it. But it was interesting enough that I started thinking about the null hypothesis, and what it means, and how statistics is changing so quickly that “the null hypothesis” we learned about in our classes may soon feel like an odd artifact…
Every year I offer my top picks for summer intensive training courses on research methods, and usually I point to several universities for the sake of options and a full range of methods. Well, this year I am pointing to just one. I looked at the course listings at the University of Michigan’s Survey Research…