As the Spark Summit East 2016 event drew to a close on Thursday, energy and enthusiasm for the topics and technologies addressed at the gathering were still high, with attendees still engaged as they headed for home and other business engagements.

Wrapping up their coverage of the event, Jeff Frick and George Gilbert, cohosts of theCUBE, from the SiliconANGLE Media team, met with a frequent guest of theCUBE. Rob Thomas, VP of product management and analytics at IBM, sat down to talk with theCUBE about something concerning virtually every corner of the tech world these days: Big Data and how to handle it.

Data, devices and IoT

One of the first points mentioned in the discussion was IBM’s expansion of its Spark team, along with its plans to work with the Watson team in exploring and developing further implementation of both utilities. Also examined was IBM’s Open for Data initiative, with a spearhead focus on the idea that the strategy behind data analytics starts with open source and new tools for streaming analytics on endpoints.

A large part of the push behind these developments is linked to a market examination of the ways that will be needed to adapt to the Internet of Things (IoT) and the vast increase in connected devices that it brings. Pre-processing, pre-analytics and addressing the unusable scaling of data shuttling are some of the ways IBM is looking to manage the data bulk sent and received by IoT devices, keeping a clear goal of how to move as much of this processing to the edge as possible.

While Thomas praised the efficiency of GE’s vertical integration, he also noted that it is not a model that applies to most companies. For most of the market, he said, the question is: “How is the average company, or the average developer, going to access data at an endpoint?”

Keeping it simple

IBM’s current approach to handling this, given the difficulty of scaling traditional data-management models to the billions of new devices informing the networks, is an emphasis on using software components of the IBM Streams technology to collect data, ingest it, filter it and build models on top of that with programming languages to determine what to send back.

As any processed data will need to be brought back to where a networked company is doing its centralized processing, the “hub and spoke” model is one that simply is not suited to the workload of today and the near future. As Thomas put it, “We’re trying to invert the approach to analytics to say ‘analytics should be decentralized.’”

Thomas also noted that IBM’s goal is to avoid building “to an enormous scale,” instead aiming to “build to the right scale.” Adding that a broader platform for IoT analytics is “definitely [IBM’s] vision,” he seemed positive on the company’s ability to handle the challenges in this field for the coming decade.

Watch the full video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit East 2016. And join in on the conversation by CrowdChatting with theCUBE hosts.

Photo by SiliconANGLE