Questions about the viability of open source abound. But during a recent CrowdChat hosted by theCUBE, SiliconANGLE’s Media team, in conjunction with theCUBE’s live coverage at Hadoop Summit 2015, these questions were answered by the impressive variety of use cases and solutions that were featured across the event.

Has Hadoop finally crossed the chasm into the enterprise world? Guests including Bloomberg R&D Guru Matthew Hunt, EMC Senior Advisor Aiden O’Brien, and IBM VP of Product Development, Big Data and Analytics Joel Horwitz weighed in on the future of Hadoop, while representatives at companies ranging from Pentaho Corp. to Talend, Inc. demonstrated how Hadoop is being implemented in industries around the world.

Spirit of innovation: Hadoop’s home-court advantage

According to John Furrier, founder and CEO of SiliconANGLE, a spirit of innovation seemed to permeate the Hadoop Summit 2015 event, from the keynote by Rob Bearden, CEO of Hortonworks, Inc. to the way even established companies like IBM and Bloomberg are using open-source platforms to seed new ideas.

United HealthCare Services, Inc. highlighted how data is improving its agility and responsiveness — which, in its case, saves lives. Other companies are using the new digital supply chain to reorganize and discover new revenue, according to an interview on theCUBE with Hortonworks president Herb Cunitz.

Later at the Summit, IBM VP of Product Marketing Joel Horwitz echoed this point on theCUBE. He said that his goal is to help customers “create that system that bridges” the traditional systems of record and systems of engagement, resulting in “systems of insight,” an innovative way of using data to further business goals.

For Bloomberg R&D Guru Matthew Hunt, Hadoop’s biggest advantage is its ability to simplify and consolidate development processes, spurring innovation.

“One exciting thing now is the power of consolidation,” Hunt told theCUBE. “This only opened up recently. The systems for ticker reporting is different from the systems for historical analytics. They don’t have to be separate any more. We’re seeing more opportunity to consolidate on fewer systems and still deliver breadth of functionality.”

Given that Bloomberg has nearly 4,000 engineers, abstracting the infrastructure away from the developers simplifies innovation and allows for changes to be implemented smoothly — even across tens of thousands of machines.

Open source: The quest continues

The questions surrounding open source — is it commercially viable, what should the business model look like, will it ever go mainstream — were not far from anyone’s mind during the event.

Peter Goldmacher, VP of Strategy & Market Development at Aerospike, Inc., weighed in with his thoughts, suggesting that a lot of open-source technologies are simply lower-cost alternatives to proprietary software. These technologies get produced in response to what some feel amounts to decades of “abuse of the community by vendors,” but 99 percent of the code comes from the CO of the open-source project, not the community, and most contributions go to sales and marketing, not code.

Anand Venugopal, product head of StreamAnalytix Impetus Technologies, Inc., later weighed in on the issue as well, saying that open source appeals to customers on a fundamental level.

Hadoop in action: Use cases around the world

Even with the questions surrounding open source, and the fact that non-proprietary solutions are often less user-friendly, Hadoop has gained ground in the industry. Talend CMO Ashley Stirrup appeared on theCUBE to talk about how his company is designing solutions for customers using Hadoop.

One customer saw a “several hundred million-dollar impact” by reducing the number of abandoned shopping carts; another, a financial services customer, decided to fight fraud by merging Twitter data with customer information. “Clearly this is a disruptive time,” Stirrup said, “and the people who can take advantage of this to provide new products and services are the ones who will win.”

Donna Prlich, VP of Products & Solutions Marketing at Pentaho, tells a similar story. Pentaho allows users to use Hadoop in order to cleanse and transform data, preparing it for analytics and introducing a level of governance and orchestration.

For Prlich, open source is always a good thing, but she says using it successfully requires an ongoing commitment to protecting and future-proofing technology for customers.

In another interview, EMC Senior Advisor Aiden O’Brien and Field CTO Chris Harrold focused on the need for enterprise-grade platforms and the emergence of devops, saying that their goal is to bring together data, analytics and applications in a single, transparent solution.

“Analytics is the killer app that DevOps has been waiting for,” Harrold said, and the company is leveraging that moving forward.

The road going forward for Hadoop

Gartner, Inc. Analyst Merv Adrian talked about the current state of Hadoop — and the news isn’t all rosy. The most important thing, he said, is to isolate for business where Hadoop can bring them value, something that can be tough to point out when there are so many potential applications. Without a clear selling point, Hadoop may be moving into the “trough of disillusionment.”

Wikibon Big Data Analyst George Gilbert and SiliconAngle Founder and CEO John Furrier also weighed in on the future of Hadoop alongside theCUBE host Jeff Frick during the kickoff to Hadoop Summit’s third day. Furrier and Gilbert compared Hadoop to railroads — a utility that enabled innovation across the United States.

Frick cited Amara’s Law, saying that people tend “to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” But he said he felt that with all the applications being developed, Hadoop is close to crossing the chasm.

But of course, the person with the biggest insight into the future of Hadoop was Hortonworks’ CEO Rob Bearden, who emphasized the power of Hadoop to deploy data across platforms.

“Hadoop creates a central architecture that allows you to deploy data across any architecture that fits the application best – on premises, Cloud, multicloud,” he said. And that has enabled the platform to reach the inflection point his team has been working toward for five years: Hadoop is now a viable platform. All that remains is to see where the implementations will take it in the future, and what innovations it and its partners will dream up to take advantage of this step forward in data management architecture.

You can read the entire chat below:

Photos by Arenamontanus and Hampton Roads Partnership