There’s no minimizing the toll of COVID-19. It upended lives to such an extent that we now have “pre-pandemic” and soon there will be “post-pandemic” and COVID-19 will be an important defining moment in our history.

In our professional world of research, the shift has been (and will be) dramatic as well. It has affected not only how we work (from what locations, and via what technologies) but the substance of our work as well. This newsletter highlights five of those shifts, which we’ve cast as “lessons learned,” though the implications are still playing themselves out.

1. The Fraud is Frightening

Soon after the pandemic began, research froze. Studies were postponed for a year or longer while clients took a wait-and-see approach in a world that was suddenly upended. The result was that all those people who signed up to participate in research surveys walked away. There were not enough research opportunities to go around. There was much else to worry about and no rewards to be had. The panels who supply hundreds of thousands of people willing to give research input suddenly found themselves empty handed.

Unfortunately that opened up a rich opportunity for fraudsters. Organized, disreputable groups set up workrooms with hundreds of employees to fill panels with real people (not robots) posing as ordinary respondents to harvest the survey incentives. The fake respondents were expertly trained to bypass fraud-detection, which meant that when we started fielding surveys again, the volume of fraud was astonishingly high.

And it had a lasting impact. The fraud we now see is amazingly sophisticated. The “respondents” know how to answer survey questions consistently, and they write in full sentences with credible-sounding open-ends. Now we have to go through a more labor-intensive process (including watching video recordings of on-screen mouse movements and reading open-ends in clusters to detect patterns in cadence and vocabulary) to find and weed out all the fraud.

At the height of the pandemic and for some types of research, we detected up to 70% of research respondents being fraudulent. Sadly, we also saw research studies – some about the pandemic itself – being circulated with findings that were clearly not true.

2. We Need Government Data

Versta turns to government agencies for data all the time, compiling and analyzing data from the U.S. Census Bureau, the Bureau of Labor Statistics, the National Institutes of Health, and the Federal Reserve. The depth, richness, usefulness, and availability of this data is remarkable. Our own primary research (and our clients’) pales in comparison to the treasure trove of what our government creates for us. Please, clients and colleagues, understand that we all need this data. Everything we do relies on it.

The pandemic offered two reminders of how critical this government data is. First, we were quickly confronted with the inherent time lag of the data. With the breadth and rigors of research from government agencies, it takes roughly two years from fieldwork to data availability for the datasets we use. Suddenly there were no valid benchmarks for measures of physical health and financial health that were declining dramatically within our study populations. We had to make best-guess assumptions, highlighting how crucially we rely on gold-standard benchmarks every day for our work.

The second reminder came as the pandemic became linked with a wider political assault on government agencies. The NIH came under attack in some of the same ways that the 2020 Census had been coming under attack. It was deeply depressing. These institutions, built up over hundreds of years, serve and protect our freedoms, health, and livelihoods. We – and you – need these agencies. So much of what we – and you – do relies on them.

3. Keep Your Trackers

Research firms (and probably you, as well) have a love-hate relationship with tracking surveys. On the “love” side, they provide excellent training in how to conduct rigorous, replicable research. They provide a consistent source of valuable work. They teach about the differences between, and sources of, normal variation and real change. And of course, they provide an invaluable historical record of the important measures central to your organization.

On the “not-love” side: They grow enormously in data complexity with each iteration, especially when edits are made. They can get boring. And along with this, they provide dangerous temptations to overinterpret the slightest shifts because clients desperately want and expect to see change, even when cautioned that change is unlikely. Too often we find ourselves having to call non-significant shifts “directional” even when they are not.

The pandemic, however, demonstrated the value of replicable, longitudinal data for organizations beyond just the U.S. Census Bureau and National Institutes of Health. The pandemic extensively changed multiple everyday behaviors for long periods of time. The clients who were already out there measuring and monitoring were able to see these changes clearly and respond to them quickly. Recent surveys we conducted for one of our clients about mental health issues in the workplace provides one example.

4. Strategists Need Our Help

The thing we love about strategists is that they know exactly what they want. By the time they come to us, they have analyzed their business needs. They have figured out what kinds of data will help them make decisions. They know why they want research, how they are going to use it, and how they want to report it.

The problem is when strategists write questionnaires. They get so intently focused on what they need, that they ignore how to elicit valid, reliable, and sensible data from people who actually answer surveys. For example, we see strategists draft surveys with huge multiple-select grids that look like data forms the IRS might design. They ask survey respondents to work their way through endless rows and columns of options, and to think carefully about multiple dimensions and response options all at once. Often they use jargon that stumps even the most educated.

As the pandemic stretched client-side research staff and budgets, we often partnered directly with business strategists. Many of them handed us questionnaires they wanted to use. It was a lot of work trying to revise and reorganize their questionnaires so that our efforts would not fail. For the most successful projects we convinced the strategists to lay out their business questions, not survey questions. Then we designed how best to get the data that would answer their questions.

5. Thought Leadership Matters

Lastly, we learned about the importance of thought leadership. Versta Research conducts a great deal of primary research to support thought leadership, generating statistics to help clients tell their story in ways that get noticed by the consumers and business leaders those clients serve. Often we think of this research as a luxury – a “nice to have” that is second to the strategic marketing research our clients do.

The pandemic taught us otherwise. Even though research budgets froze, we noticed that many of our clients began investing more in thought leadership than ever before. Some ventured into new topics and bigger studies, and they began planning more strategically for ways that research – broadly designed to explore issues core to their mission – could support multiple pillars of their efforts.

It was eye-opening, and it makes sense. Building an ongoing foundation of expertise, and becoming a known and trusted source for that expertise can carry organizations through ups and downs. Versta Research is fortunate that so many of our clients are forward-thinking. During a long and stressful time of illness, loss, shut downs and uncertainty, it kept us busier than ever before.