Like the silos that dot the Midwest filled with grain, enterprise data silos are locking away information – frustrating users and straining technical resources.  Answering a single question can require navigating multiple legacy platforms, cloud-based applications and stand alone databases. Is it possible to break down the walls of enterprise data silos without starting throwing everything away and starting scratch?

Data silos have existed almost as long as data, but technical innovation and growth in both data volume and diversity have made things even more complicated. Until recently, business data only existed in-house, but that’s no longer the case. Organizations now store data every place that has a little extra memory to spare: on the Web, in the cloud and even on their partner’s systems.  The challenge of breaking down these silos and extracting something meaningful out of the treasure trove of data stored in them is even greater. Many businesses struggle with just determining where to begin.

Gartner suggests starting small. Instead of trying to fix every data problem in the enterprise in one monster project, the firm recommends that businesses limit their focus through “information valuation,”a term coined Gartner. “Not all that data out there in the big ocean of data has the same degree of value,” says analyst Ted Friedman. He explains organizations should initially focus their effort on the most useful or valuable data. They should make the data required to address a specific question or set of problems meaningful first, instead making the same mistakes he sees many of his clients doing—setting the scope too large.  According to Friedman, narrowing the focus of an effort is key. Once that is done properly organizations can decide on which data sources are the most important, “and where to invest resources to questions about data quality, retention, integration, security and privacy.”

Analyzing information and making it more readily available is an important aspect of IT’s job. IT must excel at this function in order for organizations to reap the benefits big data has to offer. Business has evolved. Companies can no longer wait for their IT guy to access the archives from one department and get the necessary information to another. IT departments need to radically adapt or risk turning obsolete.  Fortunately, new big data focused platforms are creating new possibilities for consolidating and rationalizing data. These tools make it feasible to store and analyze massive quantities of diverse data in a single repository that can be accessed by multiple groups simultaneously. Many offer better performance than traditional tools and don’t require expensive or specialized hardware to function.

The Dangers of Tearing Down Silo Walls

Eliminating silos has enormous value, but it also comes with risks. Once data is unified, the potential for security breaches, accidental information leaks, regulatory challenges and privacy issues may increase.   Boris Evelson, a principal analyst at Forrester Research Inc. points out that, “[T]here are all sorts of regulatory issues, conflicts of interests.  The ‘Chinese wall’ between investment bank analyst and trader comes to mind,” using a metaphor for the potential problems organizations can face.  Gartner cautions organizations rushing to make big data work for them and make sense of all the data they have stored in their silos could inadvertently create privacy issues that impact customers, partners and employees. Data governance and security become even more important as organizations tear down silos because a single incident can have big consequences.

As with most business challenges, technology alone can’t eliminate data silos or the issues that surround them. Every new generation of data management solutions promises to be a savior. Unfortunately, the result is often even more data silos –  multiple data warehouses, operational database, stand alone data marts, countless spreadsheets, shared text files and  shadow systems scattered across the enterprise. It is possible for businesses to get a handle on all the data they are creating and collecting and free it from the silos that constrain it.

Organizations that are most successful

• have a laser focus on creating business value
• know technology is only part of the solution
• use a phased approach
• establish data governance and protection strategies early

Breaking down silos isn’t easy, but the potential business benefits are enormous. Are data silos causing issues in your organization? Do you currently have a project underway to consolidate data? We would love hear about your experiences, challenges and recommendations.