Versta Research is presenting findings from a study on survey fraud at the 76^th annual conference of the American Association for Public Opinion Research (AAPOR). The conference will be held from May 11 to 14, 2021. The presentation is called Finding Fraud in Public Polls: Employing Semantic Network-Based Methods for Identifying Fraud in Online Sampling. The study was conducted jointly with Cloud Research, a firm that has been developing new approaches for detecting fraudulent survey respondents.

With online panels now being used for most public polling and survey research, it is critical that researchers develop validated measures and tools to ensure the authenticity of data being collected. “That is where researchers are struggling these days,” said Joseph Hopper, president of Versta Research and one of the study’s authors. “Online panels are now dominant, and at the same time the volume of fraud is increasing, especially over the past 12 months during the pandemic. The balance of supply and demand has shifted quickly, and the usual safeguards to detect it have not been keeping up.”

The result, Hopper says, is that a lot of companies are publishing survey research that is “clearly and demonstrably false.” One component of the study being presented at the AAPOR session is a replication of incorrect findings published by the U.S. Centers for Disease Control, along with a correction of what those findings would have been if fraudulent data had been removed.

The method developed and validated uses a semantic network model approach to create sets of words with a quantifiable associative similarity.  Weights are assigned to word-pairs based on the analysis of English language corpora, and specific objective difficulty thresholds are assigned by varying the weights between targets and response options. These stimuli are then used as measures of attention, engagement, comprehension, and inauthentic behavior in pre-survey screens. The method prevents over 70% of problematic respondents from entering surveys, has a low false positive rate, and performs nearly as well as postsurvey quality review.

The study authors are Leib Litman, of Touro College; Peter Kowalski and Joseph Hopper, of Versta Research; and Cheskie Rosenzweig, Jonathan Robinson, and Adam Dietrich of Cloud Research. “We’re impressed with the power of this new approach,” said Peter Kowalski, research director at Versta Research. “We can use it on a large scale to prevent bad respondents from entering online surveys across the vast network of opt-in panels that supplies respondents for online polls.”

The findings will be presented on Wednesday, May 12, from 2pm to 4pm EDT. Interested researchers who are unable to attend can download the full presentation deck with all of the study’s findings for free.