When Strategists Write Questionnaires
The thing I love about strategists is that they know exactly what they want. By the time they come to us, they have analyzed their business needs. They have figured out what kinds of data will help them make decisions. They know why they want research, how they are going to use it, and how they want to report it.
The problem is when strategists write questionnaires. They get so intently focused on what they need, that they ignore how to elicit valid, reliable, and sensible data from people who actually answer surveys.
For example, many strategists draft surveys with huge multiple-select grids that look like data forms the IRS might design. They ask survey respondents to work their way through endless rows and columns of options, and to think carefully about multiple dimensions and response options all at once. They write questions and answers that, for a respondent, may be a confusing mix of categories that do not always belong together. And often they use insider jargon that stumps even the most educated. Sure, humongous grids can be efficient layouts of data (and, to a strategist, intuitive). But for a survey respondent, they are daunting and too much work.
The unfortunate result is that respondents complete these surveys very quickly. A survey that should take them fifteen minutes if they are being careful and thoughtful, takes just nine or ten minutes. And because of this, many of the measures are artificially low. Faced with so many items mashed into a grid, all of which can be left checked or unchecked, survey respondents tend to review and then ignore most of them. They scan, look at top-of-mind material that seems most relevant, dutifully (and truthfully) check the boxes they have focused on, and move on to the next question.
The solution? Bring in the survey or market research specialist—somebody who pays as much attention to the front end of research. Strategists need research experts who fuss over question wording and layout and about how research design may bias results. Bring in somebody who is thinking as much about the respondent experience, because ultimately the data are only as good as the sources from which they come.
Having strategists as colleagues and clients is a breath of fresh air, because starting with the end is mind is so critical (and uncommon) for truly excellent research. But just as critical are the design, fieldwork, and analysis that are the means to that end.
