There are, without question, a number of technical challenges and cultural implications associated with the Era of Big Data than can and must be addressed by the industry at large.
It’s also important to identify the most practical uses cases that enterprises can tackle today with new Big Data tools and techniques, such as ad recommendation and fraud detection, to promote widespread adoption.
But let’s not forget to Think Big about Big Data too.
By Think Big, I mean each of us in this industry should dedicate time each day (or week or month) to, for lack of a better term, daydream about the possibilities of Big Data. Or, as my friend and Big Data pioneer Abhi Mehta of Tresata put it to me recently, the question to ask yourself is:
“With unlimited bandwidth, unlimited storage and unlimited processing, what problems can I solve?”
The fact is Big Data is capable of addressing tactical issues like placing the right ad in front of the right audience, but also has the potential, as Mehta puts it, to solve billion dollar problems. And not just business problems, but social-economic problems on a global scale.
Take the global real estate slump that helped plunge the US and much of the world into a devastating recession. New approaches to Big Data make it possible to take a bottoms-up approach to the issue, allowing policy makers and financial institutions to view and understand the mortgage mess on a house-by-house basis rather than on a regional, city or even zip-code basis. With such a granular view of the data, perhaps new, targeted, effective solutions can be devised rather than simply big brush approaches like the recent mortgage foreclosure deal.
“There is $14 trillion of [mortgage] debt outstanding in the US…debt held by everybody in the world,” points out Mehta. “To address the issue, we’ve got to be able to go to governments, to the Treasurer and the leading thinkers in the financial world and say, “Hey guys, here’s a different perspective. We can attempt to solve the problem in a radical new way, keeping in mind the best interests of the US consumer”.
Now don’t get me wrong. There’s still plenty of work that needs to done, for example, to improve the functionality and stability of Big Data approaches like Hadoop, to train Data Scientists and other Big Data practitioners to make use of these new tools, and – this is especially important — to address data privacy issues raised by Big Data. And these discussions should and must occur.
But let’s also remember to Think Big and Think Different about Big Data.
Big Data consulting and service providers have an important role to play in Thinking Big. Service providers must challenge enterprises to consider new ways of approaching data processing, storage and analytics that result in more effective solutions to pressing business and societal challenges.
If you’re attending Strata Conference next week, don’t miss Mehta’s keynote on Wednesday morning, Decoding the Great American ZIP myth, where he’ll explore this topic in more depth. If you can’t make it to Santa Clara, no need to panic. We’re bringing theCUBE for three straight days of continuous coverage. You can catch all the coverage live on SiliconANGLE.tv starting February 28.