UPDATED 13:41 EDT / NOVEMBER 16 2023


Databricks debuts AI-powered Data Intelligence Platform

Databricks Inc. is rolling out a new offering, the Data Intelligence Platform, that uses artificial intelligence to help enterprises process their information.

The company debuted the platform on Wednesday. It’s based on Databricks’ flagship data lakehouse, which integrates the features of data lakes and data warehouse into a single product. Lakehouses offer the ability to analyze structured, unstructured and semistructured records in a centralized environment with access to error prevention features as well as SQL support. 

The newly detailed Data Intelligence Platform combines Databricks’ data lakehouse with technology the company obtained through its $1.3 billion acquisition of MosaicML Inc. in June. MosaicML was co-founded by Naveen Rao, the former vice president of Intel Corp.’s AI products group, and raised funding from investors such as General Catalyst. It offered a platform for developing neural networks, as well as a series of pretrained large language models.

Databricks has used MosaicML’s technology to build a software engine it calls DatabricksIQ. This engine powers the company’s new Data Intelligence Platform. Databricks detailed several of DatabricksIQ’s components in the months that preceded its official debut this week.

One component, Databricks Assistant, allows users to query data using natural-language prompts. The software automatically turns those prompts into SQL queries or Python code. Moreover, Databricks Assistant can ease related tasks such as optimizing an existing Python script a company’s data scientists use for analytics projects.

“Leveraging AI models, DI Platforms enable working with data in natural language, tailored to each organization’s jargon and acronyms,” Databricks Chief Executive Officer Ali Ghodsi (pictured), the company’s other co-founders and Rao wrote in a blog post. “The platform observes how data is used in existing workloads to learn the organization’s terms and offers a tailored natural language interface to all users – from nonexperts to data engineers.”

Databricks detailed that its existing Unity Catalog tool has also been equipped with AI features. The tool creates a centralized inventory of a company’s internal data assets that workers can use to more easily find specific records. According to Databricks, Unity Catalog leverages AI to automatically attach tags and descriptions to data assets, which makes them easier to search.

Besides user-facing features, the company is also using AI to optimize certain backend components of its lakehouse platform. Databricks detailed that a neural network helps power the autoscaling mechanism of its Delta Live Tables extract, transformer, load tool and Serverless Jobs feature. The company is also using AI to rewrite users’ queries in a way that increases their speed.

Ghodsi told Forbes that “a lot of CEOs and executives” have already adopted the Data Intelligence Platform. The offering is also finding use among healthcare professionals. According to Ghodsi, some doctors at Tufts University’s medical school are leveraging the platform to review patient data.

The executive added that Databricks is likewise seeing strong demand for its platform’s AI development tools. The company has rented 15,000 cloud-based graphics cards to support customers’ AI projects, Ghodsi detailed. Those chips are now “at capacity” because of high usage and some customers will have to wait to access them.

That demand is translating into strong revenue growth for Databricks. After a $500 million-plus funding round in September, the company disclosed that its annualized revenue run rate now exceeds $1.5 billion. Databricks achieved the milestone after achieving year-over-year sales growth of more than 50% in the second quarter.

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy