Use This Clever Question to Validate Your Data
At first I thought this was the dumbest survey question ever.
Why ask people what device they are using to take the survey, when you know from the platform meta-data what device they are using? Yes, even lame self-service survey platforms can (and should be) capturing information about device, operating system, IP address, start time, exit time, and a host of other data that most people ignore (and should not ignore!) because it does not seem relevant to their survey.
Then I realized (well, hoped) that our friends at Amtrak were actually geniuses. They were not asking because they wanted to know. They were asking because they already knew.
Asking this question is a way to validate the authenticity and truthfulness of a respondent. If somebody is filling out your survey and offering thoughtful responses, chances are high that their answer will line up with the meta-data. In contrast, if a robot, or a person who is gaming surveys for money, is taking your survey, chances are high the answer will not line up with the meta-data. So asking this question is an excellent test. Is your respondent a real person? And if so, is she really answering your questions thoughtfully?
There is, unfortunately, a lot of bad survey data out there, and there always has been. Serious research always builds in protocols for validating data and for systematically isolating anything suspicious.
At Versta Research we now ask this question, and a few others, when appropriate. It has become one of our best, inconspicuous, ingenious validation tools.
(P.S. Amtrak is, hands down, the most civilized way to travel!)
