UPDATED 19:00 EDT / FEBRUARY 17 2016

NEWS

Simplifying Big Data: Spark experts speak out on unifying platforms | #SparkSummit

Spark Summit East is bringing together some of the biggest players in Big Data and analytics, and one of the main topics revolves around Spark versus Hadoop. Dave Vellante and George Gilbert, cohosts of theCUBE, from the SiliconANGLE Media team, questioned if Hadoop is passing the baton to Spark to help the enterprise simplify Big Data.

Various industry experts sat down with theCUBE hosts during Spark Summit East to talk about the pain points of Big Data for the enterprise. A common thread weaved into these discussions is unifying the enterprise systems with Spark to streamline Big Data and make it more valuable and accessible.

Jack Norris, CMO of MapR Technologies, Inc.

Norris talked to theCUBE about agility being more important than data collection when it comes to reducing complexities of integrating technology. He also explained how MapR is expanding its training program to include Spark certification.

High-frequency decision-making process

“If you look at type large fortune companies, it is: ‘How do we leverage our data and make decisions much faster.’ It’s that hi-frequency decision-making process. How can I have an infrastructure in place that allows this organization to be agile? It’s not the companies with the most data that will win, it’s the companies that are the most agile and making intelligent assessments.”

Investing in Training

The approach we have taken is a software product approach to training. We rolled out on demand training – it’s free, it’s on demand, you work at your own pace, it has different modules with training associated and some hands on labs. … We just celebrated our first year anniversary, and we trained 50,000 people on Hadoop and Big Data. Today we are announcing expansion on Spark classes; we have full curriculum around Spark, including Spark certification training.”

 Ali Ghodsi, CEO of Databricks, Inc.

Ghodsi spoke with theCUBE about Databrick’s vision to simplify Spark and provide turnkey solutions to solve customer problems surrounding data silos to extract value from disparate data.

Uncomplicating Spark

“We were involved in the Hadoop projects, and what we saw was that really simple operations were extremely complicated. … We want to be part of that journey to simplify. So if companies can extract insights more easily, it lowers their costs, and they can do great things with it.”

“One of the visions of Spark, and what we provide is how do we give them something that works end to end, so its turnkey, rather than having to stitch together different pieces. We use the term unifying. So it is unifying different use cases.”

Unifying use cases

“Unifying these [use cases] is where you provide value to the customer — not having the siloed data and being able to unify the use cases companies have so that they can have end-to-end built pipelines that solve the problems and give the turnkey solutions they need.”

Matei Zaharia, CTO of Databricks, Inc. and creator of Apache Spark

The man who started the Spark explosion visited theCUBE to share the goals and vision of how Spark can connect people to Big Data.

Connecting to what people want

“We decided to focus on the thing that no one could do at all with extension architecture, and that was if you want to do more complex projects using lots of data, that is what we want to do. We design all our roadmaps and engines based on what people want, and we’ve been pretty careful not to lock people into this.”

The goal of Spark SQL, much like the rest of Spark, is to be able to connect to a very diverse set of storage engines wherever your data is, because Big Data in most organizations is distributed in many storage systems, and because its big by definition, it is really hard to move. So our goal is to connect to those and let you do queries across systems.”

Check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit East 2016. And join in on the conversation by CrowdChatting with theCUBE hosts.

Photo by SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU