

Live from Spark Summit 2017, theCUBE will be broadcasting exclusive interviews from next month’s open source community event. As Apache Spark continues to tackle some of the biggest data-wrangling obstacles in enterprise information technology, the platform is expected to only increase influence, closing complexity gaps in the Hadoop open-source ecosystem.
Yet Spark faces its own challenges as it scales enterprise information technology environments, spurring community efforts to address potential bottlenecks and other performance pitfalls. Ongoing enhancements for intelligent automation and code consolidation could help Spark circumvent Hadoop’s fate, as many top industry players boost support for the open-source platform.
Most recently, Cloudera Inc. launched Altus with integrated support for Spark, a platform as a service for data engineers to access on-demand, elastic infrastructure to streamline data pipelines’ creation and management. Databricks Inc., a well-funded startup founded by the creators of Apache Spark, has also been busy simplifying Spark for big data analytics.
Nevertheless, intelligent automation remains a top challenge for the Spark community, according to Wikibon analyst James Kobielus. With cognitive computing becoming an influential trend in enterprise IT, artificial intelligence and machine learning hold the promise of helping to automate out some of the complexities facing Spark.
“One of the current challenges for Apache Spark as a community is defining a clear, bounded set of use cases where it supports the AI and deep learning pipeline alongside emerging DL de facto standards, such as TensorFlow,” Kobielus stated. “We see distributed, parallelized, in-memory DL-model training as a use case where Spark has already gained adoption as a strong complement to TensorFlow, Caffe2, MXNet and other broadly adopted DL tools.”
How will Apache Spark face these challenges? SiliconANGLE will get the inside scoop at Spark Summit West, broadcasting live from the roving news desk, theCUBE. During the event, theCUBE hosts will talk with industry experts about the future of Apache Spark, how to use the Spark stack in a variety of applications, the best practices for deploying Spark at scale, and use cases from leading organizations solving big data problems.
Keynote speakers for Spark Summit 2017 include Matei Zaharia, co-founder and chief technologist at Databricks; Eric Siegel, founder and author for Predictive Analytics World; Ali Ghodsi, chief executive officer and co-founder at Databricks; Christopher Ré, associate professor at Stanford; Michael Greene, vice president for Software and Service Group at Intel Corp.; Ion Stoica, professor at UC Berkeley AMP/RISELab and executive chairman at Databricks; Matt Fryer, VP and chief data science officer at Hotels.com; Michael Armbrust, software engineer at Databricks; Tim Hunter, software engineer at Databricks; and Wes Kerr, senior data scientist at Riot Games Inc.
There are various ways to watch all of theCUBE interviews that will be taking place at Spark Summit 2017, including SiliconANGLE TV and YouTube. You can also get all the coverage from the event on SiliconANGLE.
You can watch all of theCUBE’s exclusive interviews and commentary from Spark Summit 2017 on the dedicated SiliconANGLE TV page.
All of theCUBE interviews from Spark Summit 2017, which runs until June 7, will also be loaded onto SiliconANGLE’s dedicated YouTube channel.
SiliconANGLE also has podcasts available of archived interview sessions, available on both SoundCloud and iTunes, which you can enjoy while on the go.
Stay tuned for updates on the complete theCUBE schedule and guests for Spark Summit 2017.
(* Disclosure: Some segments on SiliconANGLE Media’s theCUBE are sponsored. Sponsors have no editorial control over content on theCUBE or SiliconANGLE.)
Support our open free content by sharing and engaging with our content and community.
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.