Survey Says: Call Me on My Cell Phone
The latest data from the CDC’s National Health Interview Survey show that one quarter (25%) of U.S. adults do not have land-line telephones in their homes. So if you conduct a traditional random-digit-dial (RDD) phone survey, you will automatically be excluding one quarter of the population. Does it matter, given that surveys rarely interview everyone anyway? Probably. If those 25% are different from the remaining 75% in important ways, then excluding them will skew your survey findings.
This shift in telephone usage is critically important to survey research, because telephone surveys have set a gold standard for rigorous research over the last two decades. Given the trends, most survey researchers would agree that cell phone sample must now be included in the most rigorous research designs for an RDD survey. But doing so introduces new difficulties in design, sampling, weighting, and cost, and our collective experiences are not yet sufficient for the industry to agree on a set of “best practices” when it comes to including cell phones in survey research.
Our leading industry association, AAPOR (The American Association of Public Opinion Research) recently published a comprehensive update from the AAPOR Cell Phone Task Force that offers a good overview of the issues, complications, guidelines, and considerations every researcher should take into account when designing a telephone survey of the general population. Briefly, it covers and highlights the following issues:
Coverage and Sampling: It is increasingly difficult to reach young men and minorities via landlines, so cell phone coverage is critical. But using dual and overlapping sampling frames (both landline and cell phone) introduces new methodological complications in sampling.
Nonresponse: Response rates are lower among cell phones users, though response rates among landline users continue to decline as well.
Measurement: Data gathered via cell phone interviews is generally not of lower quality (despite concerns about audio quality, talking in public places, and distractions from multitasking).
Weighting: If dual and overlapping sampling frames are used (to include both landlines and cell phones) then data need to be statistically weighted to account for the complicated differences in probabilities that each person in the population will be recruited into the survey.
Legal and Ethical Issues: Including cell phones in surveying introduces a new set of legal, regulatory, and ethical considerations regarding auto-dialers, texting, caller ID, and do-not-call lists.
Operational Issues: Cell phone interviewing requires a unique set of protocols for recruiting, call-backs, and interviewing, all of which require special training for the people conducting the interviews.
Cost Issues: Including cell phones at least doubles the cost of a survey, and many times may triple or quadruple the cost.
Should you include cell phones in your survey? It depends on who you are trying to reach and for what purposes. When it comes to effective surveying, there are no absolutely right answers about cell phones vs. landlines, just as there are no absolutely right answers about online vs. phone surveys. Moreover, the rapidly changing landscape of how people communicate means that the best answer today may be different in six months.
Need some help thinking through the options and figuring out the best approach for your research? Feel free to give us a call. We have decades of experience and are committed to helping our clients take full advantage of important changes and innovations in research.
—Joe Hopper, Ph.D.
