When to Use Paper Surveys
One might think that with evolving technologies, the old-fashioned ways of doing research, like using paper and pencil, might disappear. But it turns out that paper surveys are not dead, and here is a nice example that dramatically illustrates their value.
As newly-appointed director of market research for the American Marketing Association in Chicago, I oversee efforts to develop and organize events related to market research. I also oversee efforts to measure, track, and analyze customer satisfaction for all other events and activities sponsored by the AMA. For the past few events, the AMA collected feedback via an online survey sent by e-mail immediately after the events. They were also using a less-than-stellar do-it-yourself survey tool that promises to deliver “actionable insights!” at the click of a button.
The results were abysmal. For every 50 attendees, we were lucky to get one response, which meant our final sample sizes ranged from one to four respondents. Even a magic “actionable insights” button can’t do much with that. For good reasons, we did not want to send out multiple e-mail reminders, and there was no money or staff to reach out with telephone calls.
So what’s a market researcher to do? First, we on the market research committee joked about doing multivariate analysis with a sample of n=1. OK, not so hilarious. Then we decided that for the next MR event that we were organizing, we would go back to basics. The plan: Distribute surveys by hand during the last ten minutes of the event; make an announcement about it; explain why feedback is important; tell people how the data will be used; ask them to do it; and collect their surveys as they headed out the door.
We considered offering a raffle incentive as well, but we didn’t need to. We got responses back from two-thirds of attendees. True, we lost some advantages of doing an online survey. The process was less anonymous, and we were stuck having to enter all the data into a computer by hand. But we got robust data that we can work with, analyze, and develop a story to share with our client.
Sometimes paper surveys make a lot of sense, and in this case it made sense because our respondent base was right there, a captive audience in one room. Evolving technologies don’t always replace the old. Telephones, computers, the Internet, e-mail, social media, and smartphones are not necessarily replacing paper surveys, but rather opening new opportunities and finding their appropriate niches. Each mode has an important and valuable place in market research. The key is to know when to use each and for what purposes, because all you really care about is getting information to make smart decisions.
Need help sorting through all the options? Feel free to give us a call. We can help you optimize your research plan by choosing the best methods, whether old or new, and then putting the best minds to work in answering your critical questions.
—Joe Hopper, Ph.D.
