With all the huge changes assaulting us every day — cloud and social computing services like Twitter, Facebook and Google+, mobile computing, cyber bullying, NSA spying and the erosion of privacy, etc. — for most people Big Data may seem like something for the data geeks. That is a mistake. Big Data sits at the center of all the technology-driven forces smashing into all our business and personal lives. More than any of these, Big Data will drive the changes that will push us all, like it or not, into a new century that will be as different from what we have known as the 20th Century was from the 19th.

Big Data impacts all of us in many ways. On a personal level, if you are concerned about the NSA revelations or about how companies ranging from Facebook and Google to your grocery store and bank are using your personal information, that’s Big Data. And you should be concerned about those things. While most Americans today are conscious of the NSA, few realize that numerous private businesses are also gathering huge amounts of data on every one of us. That includes financial data but also personal data coming off social media sites. Our cellular carriers are tracking where we go through the GPS units in our phones and may sell that data to third party businesses, for instance.

And retailers track our activities directly. Those “discount cards” that stores issue and ask for every time you check out a purchase allow the retailer to track what you buy. That plus social media data can help companies pinpoint your buying habits, what you need and don’t need, and what products or services you may want. That may sound spooky, but it also means that when your grocery store sends out a set of coupons they are much more likely to be for products you actually want rather than just a set of random guesses.

Cellular carriers and other service companies are harnessing Big Data to better understand how their customers want to communicate with them and in particular where their services frustrate users. So if the next time you need to discuss a problem with a utility the whole process seems much faster and less frustrating, you can thank Big Data.

Personally I like most of that. Not the NSA stuff which strikes me as boiling the ocean at best and moving dangerously close to a surveillance state at worst. But better, more personalized service and ads for things that actually might be useful to me are good things.

Business opportunities

But that is only part of the deal. Big data is going to impact how nearly every one of us works. For some this may just mean the availability of more information. For others it may have a major impact on their basic business processes. And some will find whole new careers with vastly better opportunities than what they are doing today. And many of those opportunities have yet to be created and cannot even be imagined today.

For instance business executives have always made major decisions “by the seat of their pants”, basically applying logic and experience to make an educated guess. This is not a good way to make decisions — even the best executives seldom bat anywhere near 50 percent. But until recently they had no choice, no better way existed.

Now Big Data analysis is being applied to change all that. Instead of guessing what your customers want, or what bothers them about your company’s products and services, Big Data analysis is being applied to find out what customers actually think. Executives who grasp this, educate themselves on what services are out there and how they can best apply Big Data analysis to the problems they need to solve, will have a huge advantage over those who do not use these tools. Over time the executives who apply Big Data will prosper, do a better job, and presumably earn promotions, while those who do not will not. And many of the new applications and opportunities will come from educated business users who understand the specific problems they face and see ways to apply Big Data analysis.

On a larger scale, we cannot afford to leave the critical decisions about how Big Data should be applied to the technologists, CEOs or politicians alone. We need to have a voice, whether that is inside our organizations or in the larger society. As with all technological innovations, Big Data can be used for good or ill. It can become the basis for new products and services and, on a larger scale a more just society that better fulfills the real needs and desires of the population, or at the opposite extreme it can be used to manipulate, blackmail, or even create a surveillance state on the model of East Germany where everyone lives in fear. Most likely it will result in something in the middle, but one thing is sure, we cannot go back to the 1960s. Unless we educate ourselves to this new force that is reshaping all our lives, we are just along for the ride. Opportunities are there for us all if we are willing to seize them.

To learn more about Big Data, including how it is being applied today, watch BigDataSV 2014 on TheCUBE today and tomorrow, February 12-13, on siliconangle.tv.

photo credit: h.koppdelaney via photopin cc

photo credit: Marco Bellucci via photopin cc