We rarely offer predictions about where the market research industry is headed. But here is a prediction I am willing to make: Soon (and I hope very soon) the year 2020 will be known as the year that fake data became a pandemic.
We see fake data flooding into our studies like never before. And even worse, we see fake data being reported in journals and in the media under the guise of market research. The good people doing the research don’t even see it yet. If you care about research like we do, it is horrifying.
What’s a researcher to do? First, remind yourself what makes for rigorous research worthy of being shared with your managers and with the public.
In that spirit, we are postponing an overview of what’s going wrong for a future newsletter, and for this newsletter showcase three rigorous studies from client organizations. All of it is research we helped design and execute over the last several months. With each of these studies having been widely shared and cited, we share them with you here. Versta Research is proud of the effort put into making these studies robust, authentic, and true.
Here is another thing you can do: Attend AAPOR’s 76th annual conference, being held virtually from May 11 to 14, 2021. Versta Research and Cloud Research, a technology provider, are presenting a research paper called “Finding Fraud in Public Polls: Employing Semantic Network-Based Methods for Identifying Fraud in Online Sampling”. Are millions of Americans really gargling with bleach to fight COVID-19? Of course not. But we’ll show you (and de-bunk) published(!) research that says they are.
Our weekly series of thought leadership articles in the Versta Blog has been jammed with a backlog of good ideas and scant time to write during this decidedly weird year. After more than ten years, it is the first time we have fallen behind. But fear not! We’ll catch up the next time around as we look towards the summer newsletter, and in the meantime, here are four brief articles to keep you reading:
As always, feel free to reach out with an inquiry or with questions you may have. We would be pleased to consult with you on your next research effort.
The Versta Team
What Makes Research Rigorous? Three Thought Leadership Examples
Mental Health and Behavioral Health in the Workplace: Example #1
The Standard Insurance Company commissioned a survey of workplace employees in the fall of 2019 and followed up with another survey in 2020, eight months after the COVID pandemic began. Findings from 2019 were alarming enough, with almost four in ten workers (39%) experiencing mental health issues. A year later in the midst of the pandemic, that number had increased to 46%.
From the standpoint of doing rigorous research, here are three things that made the surveys special:
- The research used validated and benchmarked measures of mental health (the Kessler K6 scale) and substance use (the TAPS 1 quick screen tool from the National Institute on Drug Use). This allowed The Standard to make strong claims in its reporting and thought leadership materials about incidence among the currently working U.S. population.
- Sampling was stratified to ensure accurate representation of industries, including manufacturing, healthcare, technology, education, retail, and so on. Doing this involved careful analysis of data from the Employment Projection program at the U.S. Bureau of Labor Statistics, which brings together data from the BLS Current Employment Statistics survey and the U.S. Census Bureau’s Current Population Survey.
- Longitudinal data and careful control over sampling, weighting, and matching over two waves of data allowed us to make robust (and startling) conclusions about the impact of COVID-19. This was tricky, as there were multiple factors shifting, including exceptionally high levels of unemployment among some groups of workers.
Race and Ethnic Disparities in Alzheimer’s Healthcare: Example #2
The Alzheimer’s Association commissioned research for its annual Alzheimer’s Disease Facts & Figures report, with a 2021 special focus on race, ethnicity, and Alzheimer’s in America. There were two surveys: one of all U.S. adults, and another of U.S. caregivers. The surveys documented the multiple and dramatic ways in which discrimination is a barrier to Alzheimer’s and dementia care for people of color.
From the standpoint of doing rigorous research, here are three things that made these surveys special:
- Both surveys included large over-samples of African Americans, Asian Americans, Hispanic Americans, and Native Americans, with sampling stratified within each group by age, gender, education, and income to ensure accurate representation of these groups. Data were matched and weighted to align with U.S. demographics based on our analysis of the most recent American Community Survey from the U.S. Census Bureau.
- The research used two types of sampling and sourcing: (1) probability sampling via the University of Chicago’s NORC panel, and (2) non-probability sampling via other research panels for the oversample of Native Americans and for caregivers. It required careful parsing of the data in analysis and reporting to ensure accurate and fully transparent estimates based on the data’s source and field methodology.
- The surveys asked hard-hitting and direct questions about peoples’ experiences with healthcare. We helped the association probe issues of trust, access, discrimination, participation in clinical trials, and perceptions of how respondents’ own—and others’—race, color, or ethnicity affects care. One dramatic finding of note: Half or more caregivers of color say they have faced discrimination when navigating health care settings for their care recipient.
Women Leading in Times of Uncertainty: Example #3
Wells Fargo launched a study that focused on ways women are experiencing and taking the lead in navigating their households through uncertainty during this year’s pandemic. The first release of findings was on International Women’s Day (March 9, 2021) and focused on the increasing number of younger women who are primary household breadwinners.
From the standpoint of doing rigorous research, here are three things that made this research special:
- It included women at all levels of the socio-economic spectrum, and with deliberate focus on harder-to-reach women at higher levels of affluence. The study included a matched group of men as well, but it was designed and reported to focus fully on women’s leadership, standing on its own without reference to gender differences as other studies of women and finance tend to do.
- The sampling for the study was stratified by age and by levels of assets, then matched and weighted to data we analyzed from the Federal Reserve Board’s triennial Survey of Consumer Finances. This is an important data resource not widely utilized in the market research industry. It helped ensure a balanced and accurate representation the population for reporting and communications outreach.
- The survey was designed to offer a 360-view of women’s experiences, probing not just their finances, but their attitudes, behaviors, and needs in six other areas of their lives: family, work, health, education, social, and community. In the findings released so far, Wells Fargo has focused on finance, family, work, and education, as they work to educate financial professionals around “the totality” of women’s lives.
Stories from the Versta Blog
Here are several recent posts from the Versta Research Blog. Click on any headline to read more.
Designing, generating, and tailoring data to your purpose makes a difference. Here are three examples of what that means, and why it matters.
The easiest, fastest, and most powerful way to get Census data is to download the raw data files that you can work with directly on your own.
Usually we recommend against pie charts, but here is an example with two pie charts that are better than alternative data visualizations.
AAPOR issued an urgent call to action as some U.S. politicians try to stop the Census Bureau from completing a full and accurate count of all Americans.
MORE VERSTA NEWSLETTERS