Survey Fraud Is Eerily Like AI Synthetic Data. So How Will We Know the Difference?
I became curious last week about what happened to those research people charged with fraud last year for selling fake panelists to complete market research surveys.
One by one they are pleading guilty. But there was something much more interesting for me than learning about their guilty pleas. It was the “thought experiment” it provoked while reading the US District Attorney’s press release describing the crime.
Having read so much over the last few weeks about artificial intelligence, market research, and the possibility of using synthetic AI-generated respondents to complete surveys, my human brain could not help but swap in the word “consultant” every time I saw the word “conspirator” and “AI agent” every time I saw the word “ant.”
Give it a try. In your mind, as you read this (names have been redacted), change “conspirator” to CONSULTANT and “ant” to AI AGENT:
There are weird similarities worth pondering. As we face the future of AI-generated research, we will be pretending that AI respondents are legitimate survey takers. Consultants will train the AI agents on how to take our surveys. AI agents will take large quantities of surveys (and very long surveys) that consultants say solve the problems of small sample size and respondent fatigue. All of this will presumably produce market research data, and over the coming years consultants will bill us many millions of dollars for it.
Is it so different from what the District Attorney described above? One big difference is that today’s consultants are being upfront about their data being “fake,” so ideally nobody is being defrauded. But will all that fake data accurately map onto the reality we are trying to measure?
I am open to (and fascinated by) the idea that AI synthetic data might be a valid source of insight into real human behavior. But for now I am skeptical because it seems far too similar to the fake data that our industry is already plagued by.
—Joe Hopper, Ph.D.
