Yes You Can Launch Surveys over Holidays
For some reason, nearly everybody in research believes that holidays are a bad time to ask people if they will participate in research. I am reminded of this because at Versta Research we have some big studies that launched just before Thanksgiving, and we have some big ones coming up just before and during the week of Christmas and New Year’s Day.
Just in time to spur more internal discussion, somebody posted a query about this topic on the AAPOR discussion board: “Does anyone know of any research or evidence that shows the impact of the end of year holiday time frame on data collection response rates for mail and phone surveys?” Several AAPOR members (many of them well-known, senior researchers in the field) jumped in:
“I do not have published empirical evidence to point you towards, but I and many others I have spoken to over the past four decades have always done our best to avoid starting a survey field period during the last two weeks of December and the first week of January.”
“I have spoken to different online panel companies over the past years including B2B, and they all told me to avoid Thanksgiving and the end of year holiday to field surveys. I never saw any numbers, but different companies all agreed on avoiding these periods.”
“I’ve often heard this too, and wouldn’t want to field the last week of December for a short-duration study. But it seems like a great example of something that sounds reasonable and is widely “known,” but where the knowledge may be based more on shared intuition than empirical evidence.” [emphasis added]
True to that last point in bold, it seems that all of this collective “wisdom” is not borne out by the data.
First, several chimed in with examples of their own research over holidays that performed better than non-holiday periods. One cited evidence from the U.S. Census Bureau: “Various household surveys such as the Current Population Survey and American Time Use Survey are in the field every month. Their response rates are pretty consistent throughout the year. The season or month does not have big effects on response rates, at least for these high-quality, high-effort studies.”Then Professor Mary Losch, at the University of Northern Iowa, reminded us of an article she published in Public Opinion Quarterly fifteen years ago on this topic. She looked at data from the BRFSS (Behavioral Risk Factor Surveillance Survey) conducted by the CDC, and concluded:
“These findings cast doubt on the universal recommendation that summer and/or December data collection should be avoided if at all possible. In other words, time of year of data collection should not be given undue weight in deciding when to conduct a survey. . . . In many cases, too much concern with the seasonal variations, given the time constraints under which many survey researchers operate, may be unwarranted unless and until additional research is conducted on other administration modes, populations, and content areas to firmly establish the existence of time-of-year effects or the conditions under which they may occur.”
It turns out that the survey we recently launched just before Thanksgiving did spectacularly well. I hope to tell you the full story in a future newsletter, because we have already hit a 20% response rate on first outreach for a mail survey. Fingers crossed for Holiday cheer with our upcoming surveys during the final months of December, as well!
