Enterprise data scientists and developers fed up with data that just sits there and doesn’t make money might take heart in a prediction from Ali Ghodsi (pictured), chief executive officer and co-founder of Databricks Inc.

“Data warehousing as we know it today will continue to exist; however, it will be transformed,” Ghodsi said in an interview during this year’s Spark Summit  in San Francisco.

Within 10 years, data warehouses and business intelligence tools built around them will become laser-focused on one objective, Ghodsi told George Gilbert (@ggilbert41) and David Goad (@davidgoad), co-hosts of theCUBE, SiliconANGLE Media’s mobile livestreaming studio. (* Disclosure below.)

“All the data warehousing solutions that you have today will add predictive capabilities or they will disappear,” Ghodsi said.

Structured Query Language is commonly used to get descriptive statistics, tables, averages, etc., but these are mostly past or present-based. In other words, they often do not provide insights until it is too late to act on them, he added.

Apache Spark SQL adds much-needed predictive capabilities, Ghodsi stated, adding that Databricks’ new Serverless Pools with machine learning take prediction further, while abstracting away complexity. These are basically self-managed, automatically shrinking and expanding pools of cloud infrastructure that interact with the Spark engine.

Serverless speeding past DevOps?

Serverless Pools allow data scientists to use machine learning models without a lot of prerequisite knowledge via simple serverless functions, Ghodsi explained.

“You cut costs significantly — you can now have 30, 100 people share the same resources, and it enables them to move faster, because they don’t have to worry about all the DevOps that they’d otherwise have to do,” he concluded.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of Spark Summit 2017. (* Disclosure: Databricks Inc. sponsored this Spark Summit 2017 segment on SiliconANGLE Media’s theCUBE. Neither Databricks nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE