Who’s afraid of the Big Bad Data?
2012 is fast living up to its billing of being the year of Big Data. Investors are in an expectant froth about the imminent Facebook IPO, an estimated $100bn flotation built on harvesting the personal likes of 900 million members (according to its newest financial figures). There’s the smaller but equally eye-popping $3.2bn (32 times earnings) valuation of a Big Data analytics firm, Splunk. And there’s what might be considered the curious acquisition by Walmart of the popular Facebook app start-up, Social Calendar. Curious that is until you start imagining what a retailer the size of Walmart could do with all that data about your best friend’s birthday, your wedding anniversary or your summer holiday plans.
Even as we continue to share extraordinary amounts of formerly private (or at least almost inaccessible) information in our Faustian bargain for better, more useful social media experiences, a privacy backlash is growing. Just last week, Sir Tim Berners-Lee, founder of the World Wide Web, warned of the dangers in putting so much personal data in the hands of just a few social networking companies, and urged Netizens to demand access to their personal data from the likes of Facebook and Google.
Of course, excitement and fear about data mining have been competing since the birth of the web. I remember a company all-hands meeting back in 2000 when my then publisher spoke excitedly about how online customer relationship management (CRM) would transform the media business and our publication in particular. (The dot-com crash put paid to the second part of that prediction.) Fast-forward 12 years and the new online analytics boom business is social-CRM – mining social network conversations, likes and follows to intuit and predict customer behaviour.
The reason for the new boom is the explosion in online data being created and shared. According to the technology research firm, IDC, data is growing at 50% a year. One recent report from the World Economic Forum, Big Data, Big Impact, boldly classed data a new economic asset, like currency or gold. As new social technologies and platforms come online, and as the Internet of Things begins to mature these streams of data will explode, and the companies that mine them will likely have even more power over our lives than ever before.
But, hold on a second. Does the Big Data revolution have to scripted like an out-take from Minority Report? Even as companies fine-tune their Big Data targeting in order to sell us more stuff in locations and at times when we least expect it, a growing movement of data scientists are asking the question: how can we use Big Data for social good?
This is the focus of Jake Porway, the founder of Data Without Borders (just renamed Datakind). A data scientist at the New York Times by day, Porway has lit a fire under the data analytics community by mobilising a small army of volunteer data geeks to help crunch solutions for social projects. In one example, Data Without Borders helped the New York Civil Liberties Union understand whether the New York Police Department was guilty of racial profiling (and whether they were working to meet arrest quotas) by analysing and mapping the publicly available arrest logs of NYPD beat officers. Its next collaborative goal is helping Chicago non-profit organisations unlock some of their Big Data answers.
What Porway is enabling in the US, Ushahidi, a crisis-mapping outfit has been pioneering globally since 2008. Ushahidi (testimony in Swahili) started mapping reports of violence in Kenya after the post-election fallout that year. Since then its team of data mappers have created platforms for tracking and crowdsourcing texts, emails along with social media updates from blogs and Twitter that have helped in countless crises, including the 2010 Haiti earthquake, the Japan tsunami as well as Libya’s revolution.
The potential for using smart data and social technologies to drive sustainability is also taking root in academia. Smart data thinkers from MIT, Northeastern University, Harvard and the Santa Fe Institute are working on a collaborative project, ESS (short for Engineering Social Systems). Some of the research projects it is considering include using Big Data modelling to predict food shortage in sub-Saharan Africa and using techniques such as expectation-maximisation to create economic systems that enable people to earn small amounts of money by completing simple tasks on their phones. Now imagine the business opportunities for forward-thinking telcos, mobile players and agricultural/food producers that focused their efforts on these types of sustainability projects?
Some of the biggest technology and engineering companies are starting to do just that. Current data-mapping projects in Rio de Janeiro by IBM – part of its Smarter Cities initiative – along with other urban partnership schemes run by Siemens and Philips (I’ll be writing in more detail about social tech in sustainable cities in a different column) all point to a small but important emerging corporate trend of analysing Big Data not just to mainline sales channels but to navigate a path to more sustainable living and business.
According to the New York Times, as the Big Data economy continues to grow, the US needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers. Let’s hope a good chunk of that expertise can be employed towards harnessing Big Data for sustainability. As Porway said in a speech at a recent Poptech conference, “What if we started using data, not just to ask what type of movie we wanted but to see but what type of world we want to see?”
Now that’s “Social” CRM we could all embrace.
This post first appeared as part of Matthew Yeomans’s The Social Business column in Guardian Sustainable Business.