theCUBE Live At Splunk.conf & Data Science Myths

Big Data analytics encompasses far more than just reporting and dashboards; it’s about unearthing insights and opportunities, and answering questions that you didn’t already know. Big Data analytics won’t be successful unless you focus on the problems you’re trying to solve.

Some persistent myths about Big Data stand in the way of organizations’ success, though. These relate not to technology but to culture and the way we use data. Let’s take a look at three of them.

Myth #1: Big Data makes you agile. Big Data won’t do you any good if you don’t understand the fundamentals of using it – like data governance, data storage and data quality. If you aren’t good at working with small data, Big Data is only going to make the problem worse.

For example, one of the most underestimated problems today is data drift. Different divisions within a company don’t know what to look for, so they ask for all the data in all possible combinations. Multiple copies are distributed across the organization.

The result is that different divisions come up with different results or waste time working on the same problems. Three quarters of management’s time is spent cleaning up the mess created by data drift. Even worse, duplication of effort can lead to poor decisions.

Another example of putting the cart before the horse is relying on Big Data for behavioral insights while ignoring transactional ones. Behavioral data is about interactions and paths between individual data points. While many people believe it’s a powerful way to improve the business, behavioral data is only valuable when it builds on a foundation of transactional data; it’s not one type of data source or the other. The end goal is to improve the customer experience in a way that leads to transactions. In other words, if the customer doesn’t buy, then behavioral insights are useless. Behavioral data is valuable to build on the insights that transactional data gives, but it also cannot stand by itself.

Myth #2: Hadoop is going to solve all your problems. Another Big Data myth also comes from a false dilemma; for the past two decades, much of the discussion around Big Data was centered on the enterprise data warehouse (EDW). When Hadoop came along, the conversation turned to whether or not the open-source underdog would replace the incumbent EDW and take over the analytics world.

But this isn’t an either/or proposition. You need both. True Big Data value comes with “liquid” analytics, where queries can be dispatched across one cohesive, interconnected and complementary architecture. It shouldn’t matter whether the information is stored in a Hadoop or elsewhere; you need to take the analytics to data. Hadoop indeed has its advantages, and it is a part of a solution, but only part.

Myth #3: Data scientists are king: But at best, far less than one percent of the people in a company will be data scientists. It’s an important discipline, but data scientists must drive an innovative vision of how to best leverage data. This vision must empower business and IT employees to dream up new ways to use analytics to provide business insights.

Everyone must be able to leverage Big Data. This isn’t about technology or analytics techniques, but rather about culture. The best technology platforms will reach only a fraction of their potential without a culture that embraces the value of analytics. Data scientists can’t do that alone.

Big Data analytics will reach a new level of importance when we are able to implement social, crowd-sourced collaboration platforms that allow everyone in a company to work together and learn from each other. Such platforms enable users share and build on each other’s work, and allow all employees – not just data scientists – to engage with the analytics process.

All in all, it’s not just about magical data scientists, a magical platform, or a magical new data source. It’s about empowering a talented team, leveraging the best technology, and using all available data to drive the business to new heights.

.

About the Author

Oliver Ratzesberger is Teradata’s SVP of Software and a seasoned executive and expert with over two decades of experience in Information Technologies. Oliver spent seven years at eBay, where he was responsible for one of the world’s largest IT Infrastructures.

photo credit: oohlarock via photopin cc