Perils and Pitfalls of Social Media for Research
It’s been four years since the AMA convened a panel of thought leaders in market research to ponder questions about how social media fits into research practices and methods. An excerpt of our discussion was published in Marketing News in 2010. Last month, another top industry group (AAPOR) released its Emerging Technologies Task Force report on “Social Media in Public Opinion Research.”
Where do we stand four years later? In practical terms, we’re still in about the same place. In 2010 we were seeing a flurry of new interest and experimentation. Since that time, many of us have used various types of social media, and for a variety of research purposes. But all of us are still struggling with issues of validity and reliability, and many of us still wonder whether the data and analyses are truly useful.
To be sure, academic researchers are now probing deeper and exploring whether and how social media can improve upon current methods, and this is reflected in the AAPOR report. But what is most striking about the report is that unlike other domains of survey research, there are not yet clear best practices for how, when, and where to use social media. Instead the report reviews the current status of how researchers are trying to use social media, whether successfully or not, and the practical, scientific, and ethical issues they face.
In brief, the topics and issues addressed in the report include:
- The types of social media currently being used for research purposes
- The current and potential uses of social media for research including pre-survey intelligence, focus groups, study recruiting, and even replacing surveys with social media analysis
- Challenges of adequate population coverage and appropriate types of sampling
- Data quality considerations such as data completeness and data accuracy
- Conceptual challenges when examining data and defining the units of analysis
- The validity and reliability of text analytics and sentiment analysis
- The legal and ethical issues involved in using such data and sharing findings with others
Their overall conclusion:
“Though a good deal of research to date has focused on an array of issues related to social media, far less is known in terms of if, when, and how such data may be fit for use in public opinion and survey research.”
So here we are four years later. The pace of technological change and human behavior is still changing at neck-breaking speed, but it takes time for the best and brightest to figure out how all of it should and will change how we do research.