For some reason, nearly everybody in research believes that holidays are a bad time to ask people if they will participate in research. I am reminded of this because at Versta Research we have some big studies that launched just before Thanksgiving, and we have some big ones coming up just before and during the…
Extremely careful researchers work hard to make sure their surveys reach broad and representative samples of their target populations. This usually involves multiple attempts to reach out via multiple phone calls, e-mails, or letters of request. Since statistical models of inference presume 100% response rates within random samples, the goal is always to get response…
In just the last thirty-five years, response rates for telephone surveys have been plummeting. In the 1980s, response rates of 70% or higher were common for well-designed and carefully executed phone surveys. They decreased into the 60% range in the 1990s. And then by the end of the decade, response rates of 30% to 40%…
Believe it or not, there are still plenty of phone surveys being done out there, especially for local needs assessments and evaluation surveys. Often these surveys are mandated by state or local governments to ensure that the right services are reaching people who need them. But phone surveys are extremely difficult to field anymore, as…
The Economist ran an article a few weeks ago from which I have borrowed the title of this post. It puzzles over British politicians railing against focus groups for message testing and trying to understand voter sentiment, while continuing to rely on them as heavily as ever. “In an era when voters are monitored, tracked…
Oddly enough, the researchers who most prize scientific rigor (our research colleagues in academia) are the last to have realized that online sampling can be treacherous. An article published in Wired magazine last week describes academic psychologists as “freaking out” because they just learned that a lot of their data, which they source from Amazon’s…
Surveys run the gamut from silly to serious, and from sloppy to scientific. Occasionally we do something silly if a client insists (though we generally advise against it). But we never do sloppy surveys, and we hope you never do them either. Lean as far as possible towards rigor and science within reasonable constraints of…
There is a “user-friendly” feature that some survey platforms now offer, and almost certainly you should avoid it. It is called auto-advance. I had an unpleasant encounter with it in a customer survey I just took. The idea is that, after clicking on an answer, instead of having to move a mouse or finger all…
If you worry that cell phones make it harder to get high quality, thoughtful, and richly detailed data with surveys, stop worrying. New research from a trio of survey measurement scientists (one at the Census Bureau and two at University of Michigan’s Survey Research Center) shows there are no differences in data quality and measurement…
I once heard from an overzealous boss with a Ph.D. in political science that measurement scales should always have an odd number of scale points – the reason being that measurement scales should always have a neutral middle. We sometimes hear this from clients reviewing survey drafts as well: “Shouldn’t every question have a neither,…