Don’t Fall for the Neutrality Trap
I once heard from an overzealous boss with a Ph.D. in political science that measurement scales should always have an odd number of scale points – the reason being that measurement scales should always have a neutral middle. We sometimes hear this from clients reviewing survey drafts as well: “Shouldn’t every question have a neither, neutral, or not sure option?”
The answer is “not always.” And more often than not, we try to avoid them. For most of the behaviors and attitudes we measure, nearly all people will have at least some opinion or proclivity or information that we want them to share. Offering a neither, neutral, or not sure option provides a way for people with weaker (or lazier) opinions to avoid answering.
And, indeed, they do avoid answering. We recently described new academic research showing that two-point scales maximize measurement reliability. One reason is that it does not allow an “easy out” via a neutral midpoint. Quoting the authors:
[In comparing] two- and three-category measures … the latter have substantially lower reliabilities. Our argument has been that the middle category in this situation introduces ambiguity into what otherwise would be a clear-cut choice. Respondents may choose a middle category in this situation because it involves less effort and may provide an option in the face of uncertainty. And consequently, three-category scales are less reliable than two- and four-category scales. Our results in the present study produce strong support for this hypothesis. Further, we found that the problem is somewhat mitigated in the use of five-category scales, which do not create the same kinds of problems; however, our results in the case of five-category scale are no more reliable than four-category versions that ignore the middle category.
We see this in our research and experimentation as well. Try this: Offer a neutral middle. Then ask a follow-up question for those who select the neutral middle. Give them another chance. Most of them, on second thought, will not remain neutral.
Here is our suggestion. As a default, write your questions without don’t know, not sure, neither, or neutral middle options. Assume everyone has an opinion and can answer every question. Then go back and carefully think through each question one at a time. Could somebody honestly get stuck because they don’t know something, or because they are in a situation where it makes sense to have no opinion? Those are the only questions to which you should probably add a “don’t know” or “neutral” option.
