UPDATED 14:47 EDT / JUNE 08 2016

NEWS

The data evolution: Spark as a universal platform | #SparkSummit

Evolution is a process that takes time — and this is especially true when it comes to the development of new data structures.

As Doug Cutting, chief architect at Cloudera, Inc., puts it, “To make something that’s widely used, you have to cross several thresholds, and sometimes that can take quite awhile.” Cutting hopes to integrate Spark as a universal platform but expects this switchover to be slower going than most technological breakthroughs. His time-frame estimate? Almost a decade.

Turns out that it’s difficult to teach an old dog new tricks even when it comes to software: “People learn new systems, they adopt them and embrace them,” Cutting explained to John Walls and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE Media team, during Spark Summit 2016 in downtown San Francisco.

“It’s difficult to give up these technologies lightly,” Cutting continued. “You want to pick up things that will last.”

Planes, trains and automobiles

And while adopting Spark in more and more verticals may take time, creating a product with longevity is more important than reinventing the system overnight.

“We’re using data to accomplish more than ever before,” said Cutting. It’s a big change that’s currently altering several industries — from financial institutions fighting fraud to pharmaceutical companies better understanding medications and farmers improving crop management

“It’s planes, trains and automobiles,” said Cutting, “It’s everywhere.” And even if the process takes time, Cutting said it’s to be expected: “These big institutions are bigger ships. It’ll take some time to turn them over.”

Watch the full video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit 2016.

Photo by SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU