Recent Findings on Phone vs. Online Surveys
A research article the Winter 2009 issue of Public Opinion Quarterly provided some useful comparisons of data quality between phone surveys and various types of online surveys. The findings are based on an experiment that fielded identical questionnaires via three survey modes, and, not surprisingly, there are strengths and weaknesses to each type of survey.
As reflected in the table above, the article is rather technical. But there are two key summary points worth learning from the study:
Phone vs. Online Surveys: Strengths & Weaknesses
One, online surveys fielded through panel providers may not fully represent the population. But we know from other research that they can come close, so the key is to understand the sources of potential bias and adjust for them as needed.
Two, online surveys fielded through panel providers may result in more accurate data than other types of surveys. There are two reasons for this. First, online panelists are more engaged in the process and interested in the topics, so they exhibit less cognitive laziness. Second, online surveys feel more private, so respondents provide answers that are more honest. This is in contrast to phone surveys in which people usually want to be nice and therefore provide socially desirable answers, even if it’s not how they really feel.
Which should you do, a phone survey or an online survey? And what type of sampling should you employ, probability or non-probability? It depends on your objectives. Each of these issues can matter a lot or not at all depending on the specific questions you are trying to answer. We would be happy to help you think through your options and the strengths of each. Even if there is no “right answer” there is probably a “best answer” within the context of your key objectives.
–Joe Hopper, Ph.D.
