UPDATED 20:26 EDT / JULY 20 2017

BIG DATA

Will serverless functions beat DevOps in race to democratize analytics?

Enterprise data scientists and developers fed up with data that just sits there and doesn’t make money might take heart in a prediction from Ali Ghodsi (pictured), chief executive officer and co-founder of Databricks Inc.

“Data warehousing as we know it today will continue to exist; however, it will be transformed,” Ghodsi said in an interview during this year’s Spark Summit  in San Francisco.

Within 10 years, data warehouses and business intelligence tools built around them will become laser-focused on one objective, Ghodsi told George Gilbert (@ggilbert41) and David Goad (@davidgoad), co-hosts of theCUBE, SiliconANGLE Media’s mobile livestreaming studio. (* Disclosure below.)

“All the data warehousing solutions that you have today will add predictive capabilities or they will disappear,” Ghodsi said.

Structured Query Language is commonly used to get descriptive statistics, tables, averages, etc., but these are mostly past or present-based. In other words, they often do not provide insights until it is too late to act on them, he added.

Apache Spark SQL adds much-needed predictive capabilities, Ghodsi stated, adding that Databricks’ new Serverless Pools with machine learning take prediction further, while abstracting away complexity. These are basically self-managed, automatically shrinking and expanding pools of cloud infrastructure that interact with the Spark engine.

Serverless speeding past DevOps?

Serverless Pools allow data scientists to use machine learning models without a lot of prerequisite knowledge via simple serverless functions, Ghodsi explained.

“You cut costs significantly — you can now have 30, 100 people share the same resources, and it enables them to move faster, because they don’t have to worry about all the DevOps that they’d otherwise have to do,” he concluded.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of Spark Summit 2017(* Disclosure: Databricks Inc. sponsored this Spark Summit 2017 segment on SiliconANGLE Media’s theCUBE. Neither Databricks nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU