The first thing you should do before starting research is figure out what question your research must answer. It’s not always easy. Most of us learn in school that we’re supposed to answer questions, not ask them. In this newsletter we offer some tips about the art of asking questions to ensure that your research is a success.
Other items of interest include:
- Communicating statistics: Gripping advice from Fast Company
- Forensic polling analysis
- People don’t lie on surveys
- A statistics puzzle: Do you really have cancer? Probably not.
- What Is an omnibus survey
- Game-changing product innovation
- When to choose an omnibus survey over a custom survey
- How to select the type of chart to use
- Recent findings on phone vs. online surveys
- Research that goes viral
- Research should focus on your customers, not on your products
- How to conduct a telephone survey for gold standard research
- When to kick out a survey respondent
- Conflicting surveys give you insight
While you are here, also take a look at recent comments from current and past clients. These are based on client interviews we conducted over the last several weeks.
Any questions? Give us a call. Struggling to find your questions? Please read on—we are happy to help you find them.
The Versta Team
The Art of Asking Questions
“If you do not ask the right questions, you do not get the right answers.”
– Edward Hodnett, 20th century poet and writer
Somewhere along the way to research becoming central to how businesses learn about their customers, the art of asking questions was lost. As a result, there is a lot of research for research’s sake, data in search of answers, and findings in search of questions. Occasionally an astute executive will ask “Why are we doing this?” If we struggle to find an answer, then chances are good that the research is off course and unlikely to provide insight. Good research is always designed to answer a central question, and every element of the research process, from beginning to end, must be focused on that question.
We recently provided counsel to a firm that was overhauling some of its key tracking research because it was getting poor data, service, and insight. Two vendors were pitching automated survey and analysis tools as a solution, and each offered some compelling features. We asked our client, “What is the question this research needs to answer? How might each tool deliver against the goal of answering that question?” It became clear that the client was three steps ahead of itself when they answered, “We need feedback so that we can continually improve.” This is the right sentiment and potentially a good reason for doing research, but it is not a question that can guide research. We advised the client that without a clear question guiding their deliberations, a quick decision to implement either tool would yield more data in search of a question, and twelve months down the road they would again feel disillusioned for not getting the “actionable insight” that was promised.
If you really want to help your business with research that matters and that delivers insight, ask and answer compelling questions. Francis Bacon, an early philosopher of the scientific method, advised: “A prudent question is one half of wisdom.” Likewise, a good question is one half of your research problem solved.
Surfacing Five Types of Questions: An Easy Process
Usually your business partners or clients have substantive questions—they just need help formulating them. The key is to get them focused not on the research process and methods, but on the business questions around which research (if they really need research!) will be designed.
We recommend a simple focus group technique to do this. Get all stake-holders in the same room for an hour and a half. Ideally you will have day-to-day managers as well as the senior decision-makers involved. Give each person five sheets of paper. The first step is to have each person brainstorm silently, writing questions on each sheet of paper. Lead the team through the process as follows:
1. The mission-critical questions. On the first sheet of paper please formulate three questions you must have answered by this research to make it worth investing time and money. Focus on your business, and what would help you grow your business or do it more effectively. Do not phrase these as survey questions or focus group questions, or something you would like to ask your customers. These are business questions that you need to have answered.
2. The nice-to-know questions. Now on the second sheet of paper write down three “nice-to-know” questions that you or other stakeholders might like this research to address. These are secondary questions that you are willing to forgo if time and budget become a constraint. These might be questions that others in the organization with alternative agendas will want to sneak into your research, or that somebody would say “While you’re at it, why not explore the widget market?”
3. The red-herring questions. List three questions that that could lead us down the wrong path if we are not careful designing the research. These might be related to business ideas that you have already decided not to pursue. Or they might be questions with answers that provide no opportunity for you to address. Or they might be relevant and important issues, but not the focus for this research.
4. The already-answered questions. Sometimes organizations fail to take stock of what they already know, and launch research that arrives at an answer already discovered in the last study. Write out three important questions that are related to the current effort that you already know the answers to, but that we might end up re-answering if we are not careful in designing this research. These are questions that you would definitely want answered by this research if you didn’t have the answers, and as such they are questions that people in the organization with only a partial view might be tempted to include.
5. The look-elsewhere questions. Finally, please list three questions that would be relevant and illuminating for us to pursue, but that could be answered by taking a second look at other research we’ve already done, or by analyzing other internal data.
Next, lead the group through a discussion. Some participants will not even realize their questions have already been answered by other research, and all will benefit from a collective understanding of what the research should be answering and what it should not be answering.
Will the Questions Have Useful Answers?
As the discussion unfolds and as each participant refines her or his list of mission-critical and nice-to-know questions, the final step is to ask them to formulate at least two possible or likely answers to the questions posed. If a question has only one likely answer, then consider the answer “known” and not worth pursuing. When each questions has at least two likely and different outcomes, then proceed to a discussion of what action the organization would take based on each outcome. If you find outcomes leading to dead ends where not much can be done even if answers are known, then cross the question off the list.
Ultimately, you will want to synthesize the worksheets generated and the discussion into a short list of the mission-critical questions and the nice-to-know questions that will drive the design of your research.
As a way to wrap up the focus group session, I like to ask each person, “If you could ask every one of your customers one closed-ended question, what would it be? Think before you answer, because you have one shot. Don’t waste it on something you already know, and don’t waste it on something so general that you can’t do anything with it.” This an effective way to learn how each member of your team has synthesized the discussion into a single, critical issue they need to learn via research.
How to Add Value: Ask Good Questions
All phases of research—design, execution, analysis, and communication of findings—should revolve around the central motivating question of the research. It is one piece of how you turn data into stories that your internal clients and managers can use. Formulating and knowing the question will ensure that your effort is on target, that it is incisive and relevant, and ultimately that the research is heard and used by your managers and their managers and their managers, and so on.
One of the best ways to truly add value to the research that you do is to listen carefully and formulate the right questions. Research is all about answering questions, which is, of course, all about asking questions. Building fancy statistical models is fun and brainy and definitely worth bragging about at your next party, but few people in your organization care much about models. They care about what answers those statistics provide to the fundamental business questions that keep them up at night.
More Stats and Stories
Here are several recent posts from the Versta Research Blog. Click on any headline to read more.
Helping people understand and really grasp the importance of a particular number requires you to relate it to their everyday experience in a meaningful way.
Two statisticians recently devised a way to test the likelihood that a polling firm’s data were fraudulent. Versta lists recommended ethical guidelines.
In a testament to respondent quality, most survey respondents tell the truth rather than gaming the system for incentives, according to recent online research.
One problem with stats in market research is that it deals with probabilities, not simple yes/no answers. But business people need to make clear-cut decisions.
This article describes how omnibus surveys work and how they save on cost versus customized surveys. Graphical illustration.
A lot of research supporting new product development is a like machine that ends up creating NON-innovation because of over-benchmarking. We suggest an alternative.
Two factors determine whether you should choose an omnibus survey: (1) The number of questions you want to ask, and (2) Who you want to survey.
The type of chart or graph you use should reflect the data you want to display and the relationship you want to show. Use this handy flow chart as a guide.
Reviews strengths and weaknesses of online surveys from recent published findings: They may be less representative, but give more accurate measurement.
Stories about research findings get e-mailed more than other articles. This article outlines characteristics of viral research findings.
An HBR article suggests that what we need is market research focused on people rather than products — social scientists are best for this.
Outlines seven key methodological steps in conducting a rigorous telephone survey of the US population.
Helpful survey and programming tip: Terminate unqualified respondents AFTER they answer all screening questions. We explain why.
When multiple surveys give different results, you gain an opportunity for understanding your topic in a way you might not get otherwise.
Recent Comments from Current and Past Clients
Versta Research helps clients formulate and answer compelling research questions. Here is what a few of those clients have to say:
“With Joe, he could kind of read our minds or get at what we were trying to accomplish. He was very thoughtful in the way the questions were developed to get at what we were really looking for. And that’s hard to do. That’s really hard to do. . . . Because we were doing this for public release, we were looking for headlines. We were looking for things that were going to grab the media’s attention that they would write about, and then promote [our company] in a positive light. And if you don’t get that from the survey, you don’t have that golden nugget. That was very different from other folks that we had worked with.”
– M.K., Senior Research Manager
“Sometimes with survey data, people put tons and tons of numbers in it. And for us it’s fine, but we have to repackage it for higher level people because that just not the way they digest the information. But Joe was able to condense that information. Sometimes he would give us two reports, one with everything, and then another version with highlights and the supporting facts for wider circulation.”
– V.P., Senior Research Consultant
MORE VERSTA NEWSLETTERS