As data and analytics become embedded into enterprise business models, it is clear that it takes a community to turn a concept into meaningful insights. Many technology leaders are collaborating with Apache Spark, an open-source Big Data processing platform, to help organizations analyze their data and use it to make actionable business decisions.

Ritika Gunnar, VP of Offering Management, Data and Analytics at IBM, joined by Dean Wampler, Office of the CTO, architect for Big Data Products and Services at Lightbend, Inc., sat down with John Walls and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE Media team, during the Apache Spark Maker Community Event held at the Galvanize Campus in San Francisco to discuss the joint project between IBM and Lightbend.

In the interview highlights below, Gunner, our Women In Tech spotlight this week, revealed valuable takeaways about the IBM commitment to the Spark community and her vision on how to integrate roles around data and analytics.

Taking open source into overdrive

Walls asked Gunnar about IBM and its interaction with the Apache Spark community. She explained that the progress has been so good that there are more (soon-to-be-announced) projects ahead.

“One year ago pretty much to the day, we were here and we announced our commitment to the Apache Spark community. And given the progress that we’ve made in the past year … we’re going to continue that momentum and put that into overdrive. We’re going to continue to contribute to the analytics operating system, as we coined it, in terms of Apache Spark and really help the community embrace Apache Spark all over. And we are going to take it up a notch. In addition to what we said about the analytics operating system, what do you think every operating system needs? Applications.

“We are going to facilitate the need to be able to create applications on that operating system and … for the analytics environment and integrated development environment to be able to build applications. And we are going to do that in a way that is very open and based on open source and facilitates all types of programming and languages and the community all around it.”

Insightfully reactive

Gilbert was interested in the paradigm shift of how an organization can use these applications successfully to enhance their businesses. Gunner provided an example of how things would look.

“We believe that our clients are infusing data all throughout their organizations. They’re building data-driven cultures, and in order to be able to create a very data-driven culture, that means more people in the organization need to have access to data. And as more people in the organization have access, they need to be able to very easily be able to create insights from that and operationalize them.

“The example I like to give is if a data science organization starts with a thousand hypotheses, and they want to be able to rapidly test those thousand hypotheses. They may find that only 10 of them have any relevance whatsoever. Of those 10, they may want to take one of them and be able to operationalize that into production in a way that can scale out, that is fault-tolerant, that has all the set of capabilities it needs. The belief is that today’s applications don’t just need to be reactive, but they need to be insightfully reactive.”

Role models

Gilbert talked about orchestrating a platform of data with analytic products, and he wondered about how the application tools fit within an organization. “There are a lot of different roles. What do those roles look like?” he asked Gunnar.

“We spent a lot of time interviewing hundreds of data professionals, and throughout that process what we found is that there are four main people or personas that are really involved in driving the data-centric culture. We’re starting with the data scientists …but in addition to that we believe that providing a culture of data means that it’s a team sport. It is a team sport, not just for the data scientists but also for the data engineer, for the business analyst and of course for the application developer.

“So those are the four primary personas or people that we see in some organizations — people may be playing multiple roles. So this notion of being able to have not only just collaboration between the data scientists themselves but across the set of professionals is something that is the fabric … in our next generation data and analytics technologies.”

Built for us

How did Gunner explain the benefits of integration between personas and establish buy-in to the idea of one for all?

“I like to describe it as built from an experience for me but ‘built for us,’ and that is a very important premise that each one of those personas that I talked about needs an experience that is very specific to how they consume data and process data. The way a business analyst would do it is very different than an application developer, but yet it needs to work and be ‘built for us’ so that together it is so more than the sum of the individual parts.”

The beauty of Spark

Walls addressed the issue of how all of this will work with legacy systems. Gunnar pointed to Spark as being the catalyst for taking the old and new and combining it.

“That’s actually where Spark comes into play and has been pretty pivotal right? Because it’s not just about new data … it’s about taking what is there and being able to bring along what is new. And so being able to use Spark as a common data access mechanism is really one of the most powerful things we’ve seen, and so that’s where we see actually a lot of Spark usage.”

To hear more from Gunnar about the Spark Maker Community, watch the interview below, and check out theCUBE’s interview with Gunner this week at Spark Summit 2016.

Photo by SiliconANGLE