Going real-time with Spark and SQL | #SparkSummit
Big Data analytics are becoming central to the success of all types of businesses. Analytics is a vital tool in managing the complex task of serving the customer while staying ahead of the competition. However, information is most valuable when it can be put to best use, and these days, that window can be very small; just minutes or seconds.
Customers aren’t going to wait for a company’s data processing to run in the background. That’s why real-time is so important. A company that can provide the right analytics, in real-time, will win out over their slower peers.
To shed some light on how Spark is helping companies process analytics in real-time, Dave Vellante and George Gilbert, cohosts of theCUBE, from the SiliconANGLE Media team, sat down with Anjul Bhambhri at the Spark Summit East 2016 conference. Bhambhri is the VP of Engineering for Big Data at IBM.
Moving forward from tradition
The discussion started off with a simple question: Why Spark SQL? Bhambhri replied that everyone wants to move into real-time these days, and Spark SQL takes analytics streaming to a whole new level. She continued, saying the traditional way was to take data and put it into a warehouse, but that job could take a long time. “These days, it’s all about real-time,” she said. She also mentioned Quarks, IBM’s open-source development tool that supports real-time applications.
IBM, she said, considers Spark to be a game-changer from an applications standpoint. That’s because Spark’s capabilities are all foundational to what an analytics application would need. She also pointed out how Spark’s abilities work together seamlessly, where other solutions tended to function in walled silos. Spark’s one foundation created what she called an “Analytics OS.”
Services and the future
Despite its many features, Spark doesn’t do everything. Bhambrhi took a moment to talk about IBM’s on-premise services to help companies with their Spark installations. Also, she mentioned they offered Spark as a Service on the cloud, to help businesses decouple compute from storage.
There remains, however, much to do. Bhambhri said that while technologies such as Spark were making things faster, they don’t solve the skill gap problem. People find it hard to get their arms around all these technologies, she said. To solve that skill gap, IBM was turning toward simplified tools to help businesses go from data to insight with almost no code. They were focused on getting these tools to companies and developers, she said.
Watch the full video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit East 2016. And join in on the conversation by CrowdChatting with theCUBE hosts.
Photo by SiliconANGLE
Since you’re here …
Show your support for our mission with our one-click subscription to our YouTube channel (below). The more subscribers we have, the more YouTube will suggest relevant enterprise and emerging technology content to you. Thanks!
Support our mission: >>>>>> SUBSCRIBE NOW >>>>>> to our YouTube channel.
… We’d also like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.