Big Data analytics are becoming central to the success of all types of businesses. Analytics is a vital tool in managing the complex task of serving the customer while staying ahead of the competition. However, information is most valuable when it can be put to best use, and these days, that window can be very small; just minutes or seconds.

Customers aren’t going to wait for a company’s data processing to run in the background. That’s why real-time is so important. A company that can provide the right analytics, in real-time, will win out over their slower peers.

To shed some light on how Spark is helping companies process analytics in real-time, Dave Vellante and George Gilbert, cohosts of theCUBE, from the SiliconANGLE Media team, sat down with Anjul Bhambhri at the Spark Summit East 2016 conference. Bhambhri is the VP of Engineering for Big Data at IBM.

Moving forward from tradition

The discussion started off with a simple question: Why Spark SQL? Bhambhri replied that everyone wants to move into real-time these days, and Spark SQL takes analytics streaming to a whole new level. She continued, saying the traditional way was to take data and put it into a warehouse, but that job could take a long time. “These days, it’s all about real-time,” she said. She also mentioned Quarks, IBM’s open-source development tool that supports real-time applications.

IBM, she said, considers Spark to be a game-changer from an applications standpoint. That’s because Spark’s capabilities are all foundational to what an analytics application would need. She also pointed out how Spark’s abilities work together seamlessly, where other solutions tended to function in walled silos. Spark’s one foundation created what she called an “Analytics OS.”

Services and the future

Despite its many features, Spark doesn’t do everything. Bhambrhi took a moment to talk about IBM’s on-premise services to help companies with their Spark installations. Also, she mentioned they offered Spark as a Service on the cloud, to help businesses decouple compute from storage.

There remains, however, much to do. Bhambhri said that while technologies such as Spark were making things faster, they don’t solve the skill gap problem. People find it hard to get their arms around all these technologies, she said. To solve that skill gap, IBM was turning toward simplified tools to help businesses go from data to insight with almost no code. They were focused on getting these tools to companies and developers, she said.

Watch the full video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit East 2016. And join in on the conversation by CrowdChatting with theCUBE hosts.

Photo by SiliconANGLE