Big-data company Databricks Inc. today made its flagship analytics platform available as an integrated service within Microsoft Corp.’s Azure public cloud.

The service, called Microsoft Azure Databricks, is designed to help customers better process massive amounts of data stored in Microsoft’s cloud, the companies said.

Databricks has grown to become one of the most recognized players on the big-data scene. The company was formed by the creators of the Spark research project at the University of California at Berkeley, which later became the popular open-source big data processing framework called Apache Spark. Databricks was founded to commercialize that software through its Unified Analytics Platform, which is analytics service based on Spark that’s increasingly being used to power modern workloads such as artificial intelligence.

In a blog post, Microsoft Vice President of Azure Data Rohan Kumar and Databricks Chief Executive Officer Ali Ghodsi revealed that Azure Databricks was the fruit of more than two years of collaboration. The executives said the companies began working on the service in response to customer requests for a version of Databricks that’s compatible with Azure. The service, introduced in beta last November, is now being made generally available.

“We experienced a lot of interest and engagement in the preview from organizations in need of a high-performance analytics platform based on Spark,” Kumar said. “With Azure Databricks, deeply integrated with services like Azure SQL Data Warehouse, our customers are now positioned to increase productivity and collaboration and globally scale analytics and data science projects on a trusted, secure cloud environment.”

Azure Databricks has been designed to help make things easier for customers. Rather than doing all the heavy lifting that comes with deploying Databricks in their own data centers, customers can simply access the service via the Azure cloud. Azure Databricks also provides greater compatibility with Microsoft’s own services.

With Azure Databricks it becomes possible to take data from other services and prepare it and process it using machine learning algorithms. From there, the data can also be streamed to other services such as CosmosDB and PowerBI, the executives said.

Azure Databricks was chiefly designed to fulfill companies’ interest in using data to power their artificial intelligence systems. To that end, the service was built with three design principles in mind. The first is enhancing user productivity in developing Big Data applications and analytics pipelines. The second principle was to build a system that could scale almost infinitely without skyrocketing costs. Third, the companies had to ensure that the new service met strict security and compliance standards for enterprises.

“Azure Databricks protects customer data with enterprise-grade SLAs, simplified security and identity, and role-based access controls with Azure Active Directory integration,” the executives said. “As a result, organizations can safeguard their data without compromising productivity of their users.”

“This speaks to the increasing power of cloud services,” said Rob Enderle, principal analyst at the Enderle Group. “Databricks is analytics at scale and this effort should put the analysis engine far closer to the massive amounts of data already being placed on Azure. The result should be a combination of higher performance and lower cost for analytics at massive scale.”

Image: Microsoft