For Better Response Rates, Try Dumping Qualtrics for Mail Chimp

If you suffer from miserably low response rates to surveys you send with your own e-mail list, here’s something to try: Dump the e-mail invitation functionality that is built into your survey platform like Qualtrics. Instead, use a stand-alone email tool like Mail Chimp or Constant Contact.

This will make your life harder, and it will double or triple the work involved in fielding your survey. But your survey invitations and reminders are more likely to reach potential respondents instead of being blocked by ISPs or local spam filters. The result could be that you double or triple your response rate, like we just did for one client.

For the last few years, our client’s response rate for a semi-annual survey was hovering below 20%. We moved the recruitment process off their survey platform for them. This time it exceeded 40%.

Here is our theory about what’s happening: Professional stand-alone e-mail tools are serious about not allowing spam. They have aggressive policies in place, and they work hard with ISPs to ensure that high volume coming from their servers is not blocked. The opposite is true with survey platforms. They have no policies against spam; they do not collaborate with ISPs to ensure a “pass” for volume email coming from their servers. The result, we suspect, is that a lot of email coming from survey platforms never even makes it to the end-user.

There are downsides to recruiting via platforms like Mail Chimp. The biggest is that for every reminder you send, you must download the survey data, and then clean out and re-load your email list to ensure that only non-respondents get a reminder. That is not hard, but it is time-consuming and tedious, and few people want to do it.

For us, the benefit of more-than-doubling a survey’s response rate easily outweighs the extra time and cost of a manual process.

Plus you need to be extremely careful about what you are sending and to whom. If you start crossing the line into spam-world, reputable companies like Mail Chimp will block you forever.

P.S. There is another excellent reason for keeping survey recruiting on a different platform from the survey itself. It helps you with data security by keeping PII out of your datasets. We’ll address that important issue in a separate article soon.

—Joe Hopper, Ph.D.