Sorry, Algorithms Are Not Insights
A persistent misperception in our industry is that technology can (or eventually will) largely automate our work. It reflects a fundamental confusion between the tools that we use versus the work that we do.
This confusion was evident in a recent New York Times article about the surprising amount of time that data scientists devote to the mundane task of cleaning data. The reporter suggests that if only our janitorial work could be automated (and not to worry—there are tech start-ups working on that!), then our fancy data tools could pump out insights better and faster than ever. Quoting from the article:
“It’s an absolute myth that you can send an algorithm over raw data and have insights pop up,” said Jeffrey Heer, a professor of computer science at the University of Washington. . . . Before a software algorithm can go looking for answers, the data must be cleaned up and converted into a unified form that the algorithm can understand.
True enough, you can’t send an algorithm over raw data and have insights pop up. But it is an absolute myth that you can send an algorithm over CLEAN data and have insights pop up. All that pops up are correlations, connections, and automated output. And then the output just sits there, waiting for a complex human brain to contextualize, interpret, understand, and communicate it. In other words, algorithms are nothing without our human brains and professional expertise to transform data into insights.
Most of my fancy statistical packages and research tools offer me silly promises of insight. So I guess its no surprise that reporters believe our world is populated with powerful algorithms, crawling the world of cleaned-up data, discovering insights. But no matter how much faster, cheaper, more beautifully, more simply, and more elegantly all of these tools help me with my data—these programs have never once served up an insight. For insights you need more than algorithms. You need smart people like you, dear reader, and like us at Versta Research.
ANOTHER ARTICLE ON THIS TOPIC:
