This PR Survey Is a Dream Come True. If Only It Weren’t Nonsense.
This data we saw last week reflects the type of data that PR professionals absolutely dream about and hope for when they conduct surveys. It shows hugely dramatic differences among the 50 U.S. states. The media relations people were able to pitch local outlets in each state with a story about how their area and readers are unique (and ranked) compared to all other parts of the country.
Unfortunately, this data, presumably from a survey about the pressures people face when quarantined at home during COVID-19, is almost certainly nonsense:
How do we know? We design and conduct surveys all the time, and analyze data from other peoples’ surveys, including census data, national health surveys, surveys about relationships, finances, shopping habits, etc. In 25 years of doing this we have never ever seen data that varies this dramatically from state to state. Even if we looked at political issues with today’s severely polarized electorate, we would not see differences like these if we compared the reddest states to the bluest states.
But you don’t need to be an expert to wonder about this data. Just think about it with a tiny skeptical voice in your head. Is it possible that couples are super happy in Connecticut (just 14% report strained relationships) while next door in Rhode Island they are super unhappy (75% report strained relationships)? Not surprisingly, there is no information from the company that conducted this “research” about how it was done. Who are the respondents? How were they sampled? How many in each state?
I wish I could tell you that junk surveys like this fail, because media outlets aren’t that gullible. But a quick search shows that this data was reported by local media in Pennsylvania, Michigan, New Jersey, Florida, Massachusetts, Oregon, and South Carolina.
The media placements for this survey reflect poorly on the media organizations who no longer bother to factcheck or ask questions about polling data. They also reflect poorly on the PR professionals who seem not to have posed a few hard questions to the client they represent. The PRSA Code of Ethics calls for PR professionals to “adhere to the highest standards of accuracy and truth … in communicating with the public.” The PR professionals here were almost certainly pitching bogus information from a junk survey that misrepresents the truth.
If you are doing a survey for public relations, by all means dream big with your headlines and ideas—it will definitely help in designing the content and approach. But please avoid crossing the line into this kind of absurdity. There is no need to peddle nonsense because there is always a compelling, true story you can tell with good data if you know what you are doing!
—Joe Hopper, Ph.D.
BONUS QUESTION: If you pull a random sample of 2,000 Americans (let’s say you’re doing a survey) how many people from Wyoming will you likely get? ANSWER: Four. That’s right. Even if you use a rigorous methodology of random sampling, you will have just four respondents from Wyoming, and no, you cannot pitch those results to the Cheyenne media!