Research Lessons from a Puppy: Why You Need Balanced Scales
Believe it or not, some people do not find puppies irresistibly adorable, as our newest honorary member of the Versta Research team (pictured here) can tell you. Elio arrived a couple of weeks ago. He is 13 weeks old and truly over-the-top adorable. He mostly sleeps in a crate next to me as I work at my desk.
When I take him outside every couple of hours for a little walk, roughly half the people we pass ignore him. Not even a smile! I still find it amazing. How can anyone resist saying hello and fawning over this cute puppy? Surely everyone agrees the puppy is adorable, right?
If I were not a survey researcher, I might be tempted to survey people with a question like this:
I would throw in that pathetically-weak and obviously-wrong “very adorable” choice so as not completely skew the data.
But wait, I am a survey researcher, and even more importantly, my walks with Elio ten times a day keep reminding me that I need to remember not everyone is like me. So if I want to ask people for their opinion, here would be a better survey question:
This response scale is balanced because it offers two positive choices and two negative choices. Plus it offers a reasonable way to differentiate stronger opinions from weaker ones. As hard as it is to believe, plenty of people will use the negative end of that scale.
If I actually did this survey, I admit I would still feel a visceral surprise and disappointment if at least 95% didn’t say that my puppy is “very adorable.” But my observations from the field remind that 95% is just too good to be true.
The fun and the challenge of research is to measure, document, and explain all the crazy, seemingly inexplicable variation within our human species. At Versta Research we do a pretty good job at it, even if I do feel incredulous when people fail to gush over Elio.
I am glad my puppy is here to teach me so many good lessons, including a reminder that surveys always need balanced response scales. But enough of the lessons for now. Come on, Elio—let’s go out and play.
—Joe Hopper, Ph.D.