UPDATED 20:20 EDT / JUNE 26 2013

NEWS

Hadoop Still Maturing : Needs Search, Security + Governance, says Gartner | #hadoopsummit

Gartner analyst Merv Adrian talked about Hadoop, its shortcomings and potential, with theCUBE co-hosts Dave Vellante and John Furrier, live at the 2013 Hadoop Summit in San Jose.

Commenting on his earlier keynote, Adrian mentioned that the main theme was maturity, “Things are getting close to being used by real people for real things,” and there have been some serious deployments.

How can Hadoop Mature?

 

When it comes to the open source community, things are visible long before they are ready for commercial consumption. Commercial companies don’t talk about a new technology until it’s close to beta. What was being talked about a year before were now a reality, but the conversation is still about how interesting and amazing things would be next year.

  • Search, Security + Governance

Search was the biggest theme and was starting to be done well, Adrian said. As there is talk about the Data Lake, “the undifferentiated pile” that data is pulled out of out, search is becoming critical. The rise of interactive analytics on Hadoop was the second big theme, but there was “a lot of misstatement of it being real time.”

John Furrier asked if applications were the big thing for Hadoop, or is it still stuck on analytics. Adrian said that “we are in the early moments of the platform being accepted as a vehicle to moving forward.” The main gaps are security and governance. “We have to have confidence about the security of the platform, that’s the number one issue,” Adrian says.  The market wants that before they can rely on it. “We need a governant platform.” Knox, he said, is addressing the security aspect up to a point. Once those gaps are filled, the next step would be an explosion of apps.

Securing standards

 

The biggest issue for CEOs uncovered by Gartner’s research comes down to IT was trust. Where data is concerned  aspects such as who owns it, provenance, security are still very unclear. Gartner customers are mainstream corporations where trust is very important. 31 percent of those stated they have no plans to initiate Big Data investments over the next two years in a survey targeting IT departments across all industries and geographies. Adrian explained that, in his opinion, the main reason was they didn’t have a real understanding of the value. Many are still asking if Hadoop is just hype, or if it brings real value.

When a technology’s value is proved, it becomes hardened. However, while at the Hadoop Summit, people are already talking about new technologies, such as Spark. “There’s always consolidation in the middle,” Adrian said, referring to the current state of Hadoop, while more innovation is hitting the market.

Trends exciting the cloud market

 

Asked about Amazon’s role in the cloud and Big Data markets, Adrian said that “web native organizations, businesses that started on the web, it is unlikely that they will go to an on-premise world. If they started in the cloud, they will stay in the cloud.”

When it comes to Amazon, it is hard to have a clear view, as they are pretty opaque. Amazon is a great place to experiment, people are doing interesting things in the cloud, but in the case of Amazon poster children, there needs to be some insight about how much of their portfolio is really in the cloud. “Amazon’s stated vision is goal is high volume, low margin provider. Most of us think about extracting margin. Amazon is determined to drive margin down, it’s really hard to compete with them,” he added

“People don’t want black boxes, unless they are sure the black box will work seamlessly, without errors,” Adrian stated.

  • Who’s the Red Hat of Hadoop?

Asked if a Red Had of Hadoop might appear, Adrian said that there wouldn’t be, “not in the next three years.”

“At every layer of the stack, there are multiple alternatives,” Adrian said. Everything is substitutable, some of the options being open source, some not. “Every distribution is a composition and there’s lots of room for more compositions. I don’t think we’re anywhere near the end of the innovation cycle.”

  • More trends to watch

Asked what he would keep an eye in the future, Adrian said he would focus on the gaps he had mentioned – security, governance, and administration. As there was more information shared in those aspects, he would need to research them more, in order to be able to advise Gartner customers about investing or not in Big Data.


A message from John Furrier, co-founder of SiliconANGLE:

Support our open free content by sharing and engaging with our content and community.

Join theCUBE Alumni Trust Network

Where Technology Leaders Connect, Share Intelligence & Create Opportunities

11.4k+  
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+ 
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.

SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.