Social Media Not Rocking Research So Far
A few years back, some researchers were predicting that social media would cause a seismic shift in how we conduct brand research, customer satisfaction tracking, innovation research, and so on. Others were not so sure, arguing that the fundamentals of data collection and analysis will always be the same. Both sides were represented on a panel of industry experts I moderated for the American Marketing Association (AMA) in 2010.
On Monday I attended an update of sorts, via presentations in Chicago sponsored by our AMA local chapter and the Chicago chapter of the Market Research Association. Annie Pettit of Research Now, an expert on social media analysis and listening, spoke about the current state of quantitative social media research. I was struck by these points she made, in particular:
1. Automated sentiment coding of social media content (text analysis and text mining) has limited accuracy and validity. Even the best machine-learning algorithms cannot untangle the layers of nuance and sarcasm and the rapid shifts in linguistic usage, especially when it comes to how people converse online.
2. The sources of social media data matter a lot. Twitter, LinkedIn, Facebook, Tumblr, and Blogger have dramatically different content, and the inferences one might draw from each can be jarringly different. This means that effective sampling and representation matter just as much as ever.
3. Brand mentions on social media reflect buzz, not brand awareness. P90X is mentioned far more often than Pfizer on social media platforms. But which do you think has higher brand awareness?
4. Demographic data on social media is lousy, when it exists at all. While people mostly tell the truth on surveys, they either omit demographic data or lie about it on social media platforms. Annie’s advice? If you need to look at demographics, do a survey.
5. Social media analysis is slow — or at least as slow as more traditional research design, data collection, and analysis. The reason is that high quality, useful social media analysis requires all the same thoughtful effort with sampling, data collection, data cleaning, analysis, and report writing. There are no short cuts, no matter what the techno wizard purveyors promise!
The verdict, at least for now, is that social media is a new source of data that is here to stay, and surely it can be mined for rich ideas and insights. But it is not better, faster, richer, cheaper, or more accurate than other kinds of research.
