General Electric Corp. declared on Monday that it has reached a significant milestone in its goal of bringing industrial equipment into the connected universe with the creation of an internal data lake that pools multiple types of information from different sources in order to simplify processing. According to the company, the centralized approach reduces the complexity involved in analytics lifecycle and thereby drastically reduces the time it takes to identify patterns across metrics.

The platform is based on technology from Pivotal Software Inc., the cloud analytics venture that spun off from enterprise storage stalwart EMC Corp, and  VMware Inc., last April. The joint venture sold a 10 percent stake to GE for $105 million shortly after its founding foundation as part of an agreement to develop integrated solutions for helping organizations make better use of their fast-growing information troves. The initiative launched with the stated goal of focusing on the aviation, transportation, healthcare, energy and manufacturing sectors, which are the conglomerate’s core markets.

GE’s massive new data lake is the result of that collaboration. Running above the scalable base layer supplied by Pivotal is the company’s own Predix framework, a data management system designed to provide a consistent environment for modern analytical services. The platform is the industrial giant’s attempt at establishing a standard for building and managing applications that take advantage of machine-generated transmissions, a cause to which it has rallied powerful allies such as Amazon.com Inc. and Accenture plc., among other top industry names.

To move things along, GE has provided a seed for the Predix ecosystem to form around Predictability,  a suite of internally developed services that makes the capabilities of the architecture available in industry-specific packages which lower the barrier to entry for customers. The lineup includes about two dozen offerings ranging from oil field monitoring dashboards to cloud-based asset management toolkits for the healthcare industry.

The company said that implementing the “data lake” model for the platform has helped exponentially speed up adoption by eliminating the need to standardize information into a common format, a task that IDC says can take up as much as 80 percent of the time invested into analytics projects.

The move is already paying dividends. GE claims its aviation business has successfully leveraged the system to combine more than 200 terabytes of flight data from 25 different carriers into a holistic view of the operational issues facing customers, an endeavor it described as an industry first. The integration enabled the subsidiary to uncover new insights that helped a number of customers, including Malaysia’s AirAsia, shave more than one percent off their annual fuel consumption.

GE expects to grow its deployment to 1.5 petabytes of operational data by 2015,  the equivalent of 10 million flights and more than 6,500 times the number it started with last year.

photo credit: gideon_wright via photopin cc