How Long Should a Survey Be?
Asking people to fill out long, tiresome, and boring surveys is a scourge of the research, polling, and survey industry. (Another is asking them to fill out a survey every time they interact with you — see There Are Too Many Surveys.) Asking people to fill out long surveys teaches them to avoid surveys in the future, and indeed we see survey participation rates continuing to decline. But more importantly if you are the one who needs to rely on survey data, long surveys result in measurably lower data quality.
In 2004, Sandra Rathod and Andrea la Bruna conducted experiments to examine the effects of survey length on response rates, drop-out rates, respondent fatigue, speed of answering, and data quality. In 2009, researchers at Survey Sampling International replicated the experiments and presented their findings at the recent 2010 ARF Re-Think Conference.
Their findings confirm what we know from the research five years earlier:
1. With longer surveys, respondents get more fatigued, pay less attention, and increase their speed of response as they progress through the survey
2. With longer surveys, data quality declines as the survey length increases (questions are skipped, open-ends are less complete, less effort is devoted to questions at the end of the survey compared to the start)
3. With longer surveys, respondents are more likely to cheat by answering untruthfully to avoid multiple follow-up questions
What is a “long” survey? The consistent answer from this research and other research in years past is that surveys over 20 minutes are too long. In our experience, you can nearly always get detailed data that gives you deep insights into your questions with surveys under 20 minutes. It is just a matter of focusing on the right questions (and only those questions) and then skillfully designing the survey instrument to answer those questions.
Need help? Give us a call. We would be happy to help you find the right focus and an efficient research design that delivers high quality data to answer your critical questions.
—Joe Hopper, Ph.D.
