The promises of Big Data are repeated constantly throughout the tech world – it saves money; it improves customer experience – but becoming “data-ready” is proving more of a challenge than many companies expected.

In a recent CrowdChat about how trends in cloud, Master Data Management (MDM) and Big Data are powering business, participants and hosts from Informatica World and SiliconANGLE grappled with this industry contradiction — how to help enterprises take full advantage of their data, and reap the rewards.

The struggle is real

The team behind Informatica World went directly for the crux of the matter by asking, “Why do so many companies struggle to become ‘data-ready’ enterprises?”

John Furrier, founder and CEO of SiliconANGLE Media, suggested that it is “organizational issues” rather than technology holding back the mass transition to data-driven enterprise. These structural problems can include lack of company-wide buy-in, problems translating raw data into business value and the sheer volume of data that enterprise-level businesses must contend with.

Several CrowdChat participants pointed out that company-wide participation and sharing in order to realize the potential gains of Big Data. Debbie O’Brien-Walery, VP of Global Communications at Informatica Corp., said that many companies are hiring Chief Data Officers (CDOs) to try to reach data-readiness, “but it will take an entire company to work together to achieve this.”

Robert Karel, VP of Product Strategy and Marketing at Informatica, added that in his experience, “Data was often hoarded by siloed parts of the business” in a “this is mine; that’s yours” manner. But in order to achieve maximum benefits, “cross-functional leadership must break those down starting with the CEO.”

Such data hoarding often arises from “poor quality of data – believing others will misinterpret the data and make the wrong decision,” replied Monica Smith-Mullen, principal solutions marketing manager of MDM at Informatica.

The Big Data journey

So how does a typical company move toward widespread sharing and usage of data? Ash Parikh, VP of Marketing, Enterprise Software – Big Data, IoT, Data Integration, Cloud, Data Security at Informatica, said, “People get started with manual approaches but soon realize that data management for small or big data is no joke.” He suggested that this lack of management is part of the reason many Hadoop deployments don’t meet their objectives for savings or revenue.

Robert Karel suggested a different path forward, saying, “Journey is the right word. No two Big Data paths need to be the same — starts with a business challenge that isn’t fundamentally being addressed the ‘old way’ and a pilot to challenge the status quo.” Starting with the business challenge provides clarity and often guides the necessary management approaches.

Why data?

In order to solve business challenges with data, senior executives across the company need to be on board. John Furrier posed the question: “If asked to explain how data is the new digital currency to a senior executive (CxO) what do you say/what proof points do you highlight?”

Anwesa Chatterjee, director of product marketing at Informatica, answered, “Data helps innovation,” and she asked, “How would the company innovate if they don’t have the data to prove what products/services they need to focus on?”

Similarly, Monica Smith-Mullen suggested that “data provides confidence, insights, deep understanding, measurement,” while Robert Karel replied that it helps businesses be more competitive in the modern, relationship-driven market. Competitiveness is “not your products and services; it’s your customer, supplier and employee relationships. It’s how quickly you can react to market insights,” he said.

John Furrier asked if there were any specific techniques to help with “data valuation” if “asked to put it on the balance sheet.” Ash Parikh answered, “One way to do this is put an amount against an opportunity lost — or a customer gained … or savings realized — it will all quickly add up and demonstrate the value of doing data management the right way.” If better data can help put the right product in front of the right customer, the value of that data can be better quantified.

Rise of the swamp thing

The danger, of course, is that a massive influx of raw data can morph into something worthy of a comic book nightmare if proper management is not in place to clean and process it. Informatica World asked, “Why do people joke about data lakes turning into data swamps?”

Robert Karel replied that this is simply the Big Data version of the old data quality/sewage metaphor. “It’s simply a cute way of reminding everyone that garbage in/garbage out will always hold true,” he said. Karel continued: “Data governance, data quality and MDM are still required for Big Data if you want the business to ever trust what you’re doing in that data lake.”

For Monica Smith-Mullen, the metaphor is more than a joke. “The lakes had become dumping grounds … sort of a Wild West mentality that originated with the promise of a data lake.” John Furrier agreed but said that lakes (or, better, “oceans”) are good repositories of unstructured data for storage and experimental purposes. “The key is: Don’t let it turn to a swamp,” he said.

In order to keep data repositories clean and healthy, John Haddad, senior director of Product Marketing at Informatica, offered some useful data management tactics. These include “universal metadata services to power semantic and faceted search, end-to-end data lineage and collaborative data governance so your data lake does not become a data swamp.”

How do companies benefit?

Once a clean data stream is achieved, it’s possible to develop a “360-degree” view of customer relationships. This can impact all aspects of a business, according to Robert Karel. “The 360-degree view that MDM enables serves every single function. Sales, marketing, customer service, support, legal, risk. Everyone needs that view to manage their part of the business,” he said. This is where company-wide buy-in becomes even more crucial. With everyone on board, teams can work from the same information to develop new insights and manage each component toward a common goal.

Additionally, this data-driven, 360-dgree view increases relevance and can improve the customer experience, as Monica Smith-Mullen brought up. It gives businesses “the ability to be relevant in every interaction and create great [customer experience] across the board.” This means things like “not sending an email for something they just purchased” or “not sending multiple copies of the same catalog.” This improves the relationship between companies and their customers, as well as the opportunity for conversions, since every interaction is targeted and potentially useful, rather than scattershot.

Managing data in the cloud

Given that many companies store their data in some combination of cloud and on-premises databases, Informatica World asked, “What are the basics of cloud data management for a hybrid cloud environment that has some data both on-prem and public cloud?”

Robert Karel said cloud and hybrid data management should follow the same basic best-practices of all data management: “Connect. Cleanse. Master. Secure. Govern. Generate Business Value. Repeat.” But storing data in hybrid or cloud spaces does add a few additional layers of challenge, as Ajay Gandhi, VP, Informatica Cloud Product Marketing, outlined: “scalability, connectivity, data visibility and operational control.”

But these challenges can be overcome by considering where data will end up and what it will be used for, Monica Smith-Mullen concluded. “It’s automating how data is managed and ensuring the data is trusted and can be used confidently without further manipulation. Where the data resides is becoming less and less important.”

photo credit: ‘Llyn Y Gader’ – Snowdonia via photopin (license)

Read the full chat below: