Survey Progress Bars Don’t Matter (Sort Of)
Progress bars that show respondents how far along they are in a survey have become standard in our industry. But are progress bars a documented “best practice” for survey research, or just another convention? A recent article in the Social Science Computer Review offers a meta-analysis of 32 field experiments designed to see whether progress bars improve completion rates.
The answer is that truthful progress bars—ones that move at a constant rate, aligned with real progress in the survey—make no difference. They do not improve completion rates nor do they improve the quality of data collected. But you can improve completion rates by manipulating the rate of the progress bar. If a progress bar moves quickly at the beginning of a survey and slower at the end, more respondents will finish. If a progress bar moves slowly at first and then speeds up, people tend to quit.
Studies also show that people like progress bars. For example:
- If given a choice, three out of four survey respondents opt to show progress bars (D. Heerwegh, 2004, “Using progress indicators in web surveys.” Paper presented at the 59th Conference of AAPOR.)
- In a Gallup survey of respondents who regularly choose to participate in surveys, most said they would recommend that survey designers use them (F.G. Conrad, M.P. Couper, R. Tourangeau, and A Peytchev, 2010, “The impact of progress indicators on task completion.” Interacting with Computers, 22, 417–427.)
- If progress bars are hidden, but made available through survey links so that respondents can check how they are doing, one-third will do so (K. Rao, and M.P. Couper, 2009, “Clarifying the ‘Progress’ of progress indicators. Paper presented at the 64th Annual Conference of AAPOR.)
Some data suggest, as well, that respondents evaluate surveys with fast-to-slow progress bars as being more interesting and shorter in duration.
So what’s a researcher to do? As Chris Harrison, a professor of human-computer interaction at Carnegie Mellon University, recently told the New York Times: “Do users really want the truth, or do they want the more relaxed, more comfortable experience?”
Given that our industry relies on the continued good will and truthfulness of people who tell us what they are doing and thinking, we at Versta Research favor the truth 100%. We recommend using constant-rate progress bars to improve the user experience (though leaving them off does no real harm) so that respondents can make informed decisions about whether to continue or to quit. Indeed, the burden is on us to make sure our surveys are relevant and engaging enough that respondents will want to finish without being tricked.