Conflicting Surveys Give You Insight
When multiple surveys about the same topic give different results, consider yourself lucky. It provides an opportunity to dissect and understand the question you are trying to answer in a way you might not get otherwise. A recent New York Times article provides a nice example when it comes to polls about health care.
Various surveys, all rigorously done, showed support for including a public option in health care reform varying from 44% to 66%. All asked neutral, unbiased questions, and all provided appropriate answer categories. But each used slightly different wording, which dramatically affected the results. Comparing the proposed reforms to the Medicare program boosted support, while referring to it as a government-run insurance plan killed support. If you are the marketing or communications team charged with the task of selling health care reform, these are exactly the kinds of conflicting results that help you. They give you deep insight about how to position your product or service, and what kind of messaging you need to develop.
Of course this example also highlights the critical importance of questionnaire design when launching a survey. Ostensibly similar efforts to measure the same thing can lead to different answers. It is easy to write questions and field surveys, but not so easy to nail down exactly what needs to be measured, and not so easy to anticipate how your audience will respond to the nuances of words and design.
The lesson? First, pay a great deal of attention to question wording, and get input from many members of your team: survey experts, outsiders, insiders, the brand team, your business executives, and so on. Second, when you get results that don’t make sense because they contradict other data, look for opportunities where this conflict can help enhance your understanding, because usually it can. If you need help with either, give Versta Research a call. We are happy to bounce around ideas and provide some initial thinking at no charge.
—Joe Hopper, Ph.D.
