UPDATED 08:25 EDT / DECEMBER 21 2015

NEWS

What you missed in Big Data: Real-time intelligence

The challenge of handling the growing amounts of real-time data entering the corporate network returned to the center of the analytics discussion last week after IBM Corp. added four new stream processing services to its public cloud. The star of the lineup is a machine learning engine that allows organizations to create pre-programmed algorithms for picking out useful information from the transmissions generated by the growing number of connected devices deployed in the workplace.

An electronics manufacturer, for instance, could use the service to immediately detect when a sensor embedded in an expensive piece of equipment signals a malfunction and automatically alert the nearest technician. IBM is touting the functionality as a way to cut through the massive volume of machine-generated signals produced every second in such environments, which can overburden not only analysts but also the technology infrastructure that supports their work. The latter problem has proven to be a particularly major hindrance for the internal business intelligence efforts of Yahoo! Inc., which last week open-sourced its engineers’ solution.

The Data Sketches library is a collection of stream processing algorithms that avoid the overhead of individually scanning every new item in a real-time feed by substituting direct counting with estimation, thereby sacrificing some accuracy for a significant speed boost. A proof-of-concept implementation Yahoo! showcased to the press was able to blaze through a sample set of 100 million values that took two and a half minutes to analyze by regular means in just under three seconds. That technology is meant mainly to help provide high-level measurements like visitor statistics, which fellow web giant Flipboard Inc. is also tackling.

The startup followed up the release of Data Sketches with the introduction of an embedded Google Analytics dashboard for its namesake content aggregation portal that aims to help publishers gain a better understanding of their audiences. The integration makes it possible to track visitors in real-time with visibility into all the usual metrics, including engagement and churn rate.

Image via Geralt

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU