How many people do you think are college-educated in the United States? If you’re like most clients we work for (whose family members, friends and co-workers nearly all have college degrees) you will substantially over-estimate that number. The answer is a mere 29%. If you include associate’s degrees, then it goes up to 37%. These…
You might think that writing a feature article about liars, cheaters, and trolls in online surveys means we have a pretty damning view of online polls as a method of research. Not so. We are all in when it comes to online polling, and in fact it comprises the majority of the survey research we…
This week one of our sample suppliers said “no” when we asked if we could re-contact respondents for a purpose other than our original request. Good thing I asked, and you know what? I’m glad they said no. I want suppliers to think carefully about seemingly simple requests, and to point out ways in which…
This survey was done by a fancy business professor, in an endowed chair, at a top-ten university, along with a managing director at one of the world’s largest consulting firms. Then it was printed in one of our industry’s top publishing outlets. It takes less that a minute to scan this data and to realize…
A good friend and client likes to joke during every project: “Remember Joe, correlation does not imply causation.” Well, I thought she was joking. Surely everybody with a job in marketing research knows this? Apparently not. Even professors at fancy business schools seem to forget it, as evidenced by this snippet of an article a…
We just completed a very cool research project (a concept test) that perfectly demonstrates the power of MaxDiff and TURF analysis. These two techniques can help you make smarter product decisions than other types of concept tests. Here is an overview of what we did and why it was so powerful. We asked consumers to…
Suppose you could have a neural network machine-learned algorithm that is 91% accurate in predicting who will buy your product and who will not, just with a computer facial scan. Wow—91% sounds amazing! Don’t buy it, and don’t be a sucker for big numbers until you look more closely at the math behind those big…
A really good qualitative researcher knows how hard (and how frustrating) it can be to get respondents to tell us things we want to know. We have our interview guide, with all our questions laid out in a nice logical progression. But often when we ask questions, respondents have less to say than we had…
Dividing a market into unique segments makes sense. But the statistical methods we rely on for segmentation often result in segments that are strongly differentiated in useless and misleading ways. A recent analysis of Facebook data by The New York Times illustrates this perfectly.
If you conduct a well-designed, well-executed survey of 500 people you probably think (and yes, we may have told you) that your margin of error is ±4%. It’s beautiful. It’s all based on math. And if you ever had that “aha” moment in statistics class, you probably agree that the mathematical proof of it is…