How to Find Gold in Your Data Mine

I’ve always been intrigued by the promises of data mining because it offers such a magical solution to much of what we do in market research.  If only we had a tool or technology that would discover hidden patterns and insights in our data.  We would not have to think so hard, or work so hard, or hire really smart people to help our clients design research, analyze data, and present findings to their executive teams.

The truth, however, is that while technology and tools can multiply our capabilities and help us work better and faster, they cannot discover meaningful patterns or find hidden insights. Only smart people can do that.  The reason is that market research data only become meaningful within a context of questions that need to be answered, or stories that need to be told.  Tools and technology cannot supply that context.

We are working with a client who has been struggling for the last five months to find a story in survey data.  They commissioned the survey to generate data for a whitepaper for presentation to business level clients and prospects.  They’ve been staring at tables and banner tabs, pie charts and bar charts, correlations and gap analyses.  But squeeze the data as they might, the story will not emerge.

Here was our counsel:  Before you can find a story in the data, be more explicit about the context that will bring it to life.  So we asked each person on the team to write three or four dream headlines that they would like to see come out of the research.  Our instructions were as follows:

1.  Focus on headlines that would be most useful to communicate to your audience

2.  Do not look at the survey or the data—forget about what you think it says

3.  Do not go back to any of your previous documents or thinking—do it top of mind, quickly

4.  Do not worry about making your headlines pretty or accurate—make it a brain dump of your dream headlines

Of course we cannot guarantee that the data will support every claim they want to make.  But we can guarantee that with so many data points and ways of linking data in even the smallest of data sets, there are compelling ways to make that data support a story that is on target and relevant.

How do you find the gold in your data mine?  By providing a frame of reference in which the data becomes gold.  Take three steps backward to consider the critical (and specific) questions you need to have answered and outline the relevant stories that would be useful to your audiences.  Then start mining your data for answers to those questions and for data points, contradictions, anomalies, and surprising patterns that relate to your stories.

—Joe Hopper, Ph.D.