Today I saw my first “dumb election statistics” article of the season, with AP News offering this dramatically titled story: Kamala Harris’ election would defy history. Just 1 sitting VP has been elected president since 1836. We are bound to see tons of headlines and stories like this during the next several weeks. It brought…
One luxury of doing primary research is that you can get data perfectly tailored to the problem you need to solve. Primary research allows you to design how data will be elicited, and how, exactly, attitudes and behaviors are measured. Hence, you get the insight you need. If you don’t have perfectly tailored data, you…
Customizing MaxDiff exercises by piping in text from previous survey answers might seem like a good idea, but analyst beware: if you are estimating individual-level scores using hierarchical Bayes or latent class analysis, you must split the data into subsets before you calculate the scores. Your results will be nonsense otherwise. Here is the scenario…
Surveys run the gamut from silly to serious, and from sloppy to scientific. Occasionally we do something silly if a client insists (though we generally advise against it). But we never do sloppy surveys, and we hope you never do them either. Lean as far as possible towards rigor and science within reasonable constraints of…
The problem with most business approaches to “analytics” is that they rely on automated, unthinking algorithms. The algorithms scan boatloads of data and generate correlations that are surprising and, therefore, presumably deeply insightful. But in most cases the correlations are not insightful. They are random coincidences, which unfortunately does not stop humans from sharing with…
We just completed a very cool research project (a concept test) that perfectly demonstrates the power of MaxDiff and TURF analysis. These two techniques can help you make smarter product decisions than other types of concept tests. Here is an overview of what we did and why it was so powerful. We asked consumers to…
Suppose you could have a neural network machine-learned algorithm that is 91% accurate in predicting who will buy your product and who will not, just with a computer facial scan. Wow—91% sounds amazing! Don’t buy it, and don’t be a sucker for big numbers until you look more closely at the math behind those big…
If there is one thing I hope you remember from your college statistics class, it’s this: Correlation does not imply causation. This is especially important to remember in our world of big data. Any large dataset will have hundreds of thousands of correlations, but most of those correlations will reflect purely random occurrences that mean…
Suppose your colleague left her job tomorrow and you needed to recreate, and double check, all of the work she produced for a research study just completed. You probably couldn’t do it. Think of every step along the way: exactly how the survey was programmed, how data were cleaned, why certain respondents and not others…
Here’s a sobering question for professionals who work in the “insights” industry: How much of your market research really offers insight? Not just validating data, or routine measurements, or pro forma tracking. Not just percentages of “how many said what” in a customized study. By true insight we mean: a direct answer to a deeply…