Microsoft embraces open-source offerings such as Hadoop, which processes big, “messy” data sets, and Spark. But it takes schooling for data scientists to use these platforms. Microsoft hopes to ease the process with its Hadoop-based infrastructure in the Azure cloud.

Bill Jacobs, Director of Analytics and Product Marketing for Microsoft, told John Walls and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE media team, during Spark Summit 2016 that Spark 1.6 is now available in the Azure Cloud, allowing data scientists to have “a fully running Spark cluster.”

A premier edition includes R Server technology. “This makes it possible for data scientists to take advantage of Spark,” says Jacobs. “There is a huge amount of performance available for the data scientist who would otherwise have to become quite schooled in Spark and Hadoop.”

While many sectors use those messy data sets, Jacobs sees industrial IoT, such as in the automotive industry, as a big growth area. “Statistical process control is not new to them,” says Jacob, noting that automakers used statistical processes beginning in 1935. “It’s a spiking industry because the knowledge of how to use statistics to improve manufacturing processes is not new. It can also improve customer experience by applying data science.”

Watch the full interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit 2016.

Photo by SiliconANGLE