UPDATED 14:32 EDT / DECEMBER 11 2015

NEWS

Wikibon’s George Gilbert defines the new, machine-learning based analytics pipeline for IoT

Capturing and managing the huge volumes of data being generated by Internet of Things (IoT) and deriving value from it requires a new data analytics architecture, writes Wikibon Big Data Analyst George Gilbert. In his latest Professional Alert, “Recipe For An IoT-Ready Analytic Pipeline,” Gilbert provides a road map for building that new pipeline, first at the IT director level and then for IT architects.

The best way to understand this emerging IoT analytic pipeline, he says, is to find elements in the traditional approach that are changing and extrapolate based on the new requirements. The cost of capturing traditional data manually has stayed roughly constant at $1 billion per terabyte for several decades. But the new IoT data is generated and captured at a marginal cost approaching zero. The new data pipeline must leverage elastic clusters of commodity hardware and software using automated management, bringing the cost of capture and management to as close to zero as possible.

The data pipeline needs to support a much higher data velocity and provide near real-time responsiveness between capturing data and driving action, while still leveraging historical data to improve the context of analytics. It needs to provide converged analytics, supporting both batch and real-time as well as both business intelligence and machine learning on any data type.

An example application is General Electric Co.’s Predix software-as-a-service (SaaS) application for predictive maintenance service for industrial equipment. This analyzes continual data streams from instrumented machinery to monitor and anticipate maintenance needs for smart, connected products operated by a manufacturer’s customers.

This “messy” data is semi-structured and often originates in analog form from sensors. The structure evolves over time, requiring flexible management. The sources are highly decentralized and in some cases (such as airplanes, automobiles and train engines) in motion. The system needs edge processing capability to separate normal readings from abnormal ones that might indicate a developing issue and send only the latter over the network, which may have low bandwidth and intermittent service.

The full alert discusses the new architecture in more detail.

Image via jeferrb

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU