The data evolution: Spark as a universal platform | #SparkSummit
Evolution is a process that takes time — and this is especially true when it comes to the development of new data structures.
As Doug Cutting, chief architect at Cloudera, Inc., puts it, “To make something that’s widely used, you have to cross several thresholds, and sometimes that can take quite awhile.” Cutting hopes to integrate Spark as a universal platform but expects this switchover to be slower going than most technological breakthroughs. His time-frame estimate? Almost a decade.
Turns out that it’s difficult to teach an old dog new tricks even when it comes to software: “People learn new systems, they adopt them and embrace them,” Cutting explained to John Walls and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE Media team, during Spark Summit 2016 in downtown San Francisco.
“It’s difficult to give up these technologies lightly,” Cutting continued. “You want to pick up things that will last.”
Planes, trains and automobiles
And while adopting Spark in more and more verticals may take time, creating a product with longevity is more important than reinventing the system overnight.
“We’re using data to accomplish more than ever before,” said Cutting. It’s a big change that’s currently altering several industries — from financial institutions fighting fraud to pharmaceutical companies better understanding medications and farmers improving crop management
“It’s planes, trains and automobiles,” said Cutting, “It’s everywhere.” And even if the process takes time, Cutting said it’s to be expected: “These big institutions are bigger ships. It’ll take some time to turn them over.”
Watch the full video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit 2016.
Photo by SiliconANGLE
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
