While several startups and smaller enterprises have been able to create enormous disruption in the field of data utilities, the major companies are most commonly the ones who get to drive discussion and market adoption of those enabled technologies.

Joseph Sirosh, corporate VP of the Data Group at Microsoft, met with John Furrier (@furrier) and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE Media team, at the Hadoop Summit in San Jose, California, to discuss Microsoft’s approach to data tools, its plans for development and the importance of partnerships.

Manufacturing app dev tools

Early in the interview, Sirosh raised the example of how machine learning can be used in India to predict when a student is most likely to drop out of school so that action can be taken to prevent them from doing so, demonstrating how powerful the use of data for real-world applications has become. He also cautioned against relying too heavily on pure mathematics, saying, “When mathematics is not that effective, when you have tremendous amounts of data, you can be unreasonably effective. And it’s not just about data, it’s about algorithms that operate on the data and can really extract useful information that can actually drive an action. … That is the transformation we are all undergoing.”

Sirosh also asked, “Why has AI so dramatically changed in the last few years compared to 30, 40, 50 years ago when it was started?” He answered his question, saying, “It’s all the power of data.” He then used the refinement of mass-manufactured clothing processes improving to the point where people can find what they want to wear instead of having it specifically tailored to them as an analogy for the possibilities now open through effective data usage.

“When you have these finished APIs, application development becomes very simple,” he said. “You just grew those applications together, those APIs together, and you get a very powerful application all residing in the cloud, supported with the SLAs of the cloud, and backed by a company like Microsoft. And that creates unreasonable speed in developing intelligent applications. That’s a huge revolution.”

Data transformations

As Sirosh highlighted, Microsoft services enable “a comprehensive layer of security that a typical enterprise running on their own, in their own data-center, will find it incredibly hard to manage and match in its competency.” In addition, the tools and services offered in conjunction with its Azure cloud utilities enable further options for the companies looking to use them. In particular, Sirosh was proud of the developments made in the database architecture.

“Here’s a transformation that happened. It’s not just a database, it’s an intelligence base, meaning intelligent models, built out of R, are managed in a database system,” he said. “It’s a management system that now extends to predictive intelligence as well. So when you bring all that together, that’s when you have the ability to build extremely powerful applications … that can take in real-time data, do real-time analytics, and drive real-time decisions.”

He added that Azure Stream Analytics “allows you to put a standing query in the flow of data and … do complex event modeling … and it does aggregate in real-time as the data flows through. And those aggregates can go to dashboards … to call machine-learning APIs and get results back. So you can do things like finding anomalies in a data stream, or machines about to fail, or do predictive analytics from the streaming data. And that is an incredibly powerful capability.”

Partners in data power

The interview concluded with a consideration of the roles played by partnering companies with Microsoft’s data divisions, with Spark being an early focus in this section. “Spark supports streaming analytics. So you can now combine there. … Spark streaming with batch analytics, with powerful machine-learning capabilities from our server, all of that, deployed in the cloud as a service,” Sirosh said.

However, as Sirosh pointed out, Microsoft is also looking at partnerships with companies like Databricks, Inc. and Red Hat, Inc. “so that we can actually create … incredible value at a platform layer for application developers.” As he noted, building new value is what Microsoft is trying to do.

Watch the complete interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of the Hadoop Summit US.

Photo by SiliconANGLE