When “No Difference” Makes a Difference

Contrary what a methodological purist in the social sciences might recommend, we often design survey questions with scales that have no neutral midpoint or don’t know option. For example, we sometimes use scales that look like this:

Rather than this:

Or this:

Why? Because survey respondents tend to over-use neutral midpoints or don’t know options. Often their opinions are weak (or ambivalent) and it is easier to pick the middle option rather than thinking through the best answer that truly represents their opinion. A large majority of these fence-sitters can and will choose a non-neutral option if skillfully probed with additional questions, and the outcome will be nearly the same as if the neutral option were not offered in the first place.

There are, however, many situations in which it is important to include a neutral midpoint. For example, in campaign polling we often want to know not only the direction of opinion, but the intensity as well. Why? Because identifying (and targeting) the pool of fence-sitters, especially the ones who lean to our side, is critical.

Another situation when a neutral midpoint is important is when neutrality itself is theoretically important—that is, when it reflects a truly meaningful idea rather than just uncertainty or ambivalence.

Here is an example from a survey we just did. Respondents were asked to review the content of a website and then evaluate the usefulness of that website. As part of the survey, they were asked:

In this case, no difference does not reflect uncertainty or ambivalence, but something important and meaningful.  In fact, our client would like to see most people reporting no difference.

As always, good research design must be driven by the unique objectives and specific questions that need to be answered.  With response scales, that means thinking about the most appropriate way to construct a measure for every question rather than relying on a library of boilerplates or thoughtlessly applying standard rules to every situation.

Need help deciding upon the best scales to use for your research questions?  Versta Research would be happy to advise you so that any neutral, don’t know, or no difference options you include will make a huge difference in the insights you get.

—Joe Hopper, Ph.D.