UPDATED 20:20 EDT / JUNE 26 2013

NEWS

Hadoop Still Maturing : Needs Search, Security + Governance, says Gartner | #hadoopsummit

Gartner analyst Merv Adrian talked about Hadoop, its shortcomings and potential, with theCUBE co-hosts Dave Vellante and John Furrier, live at the 2013 Hadoop Summit in San Jose.

Commenting on his earlier keynote, Adrian mentioned that the main theme was maturity, “Things are getting close to being used by real people for real things,” and there have been some serious deployments.

How can Hadoop Mature?

 

When it comes to the open source community, things are visible long before they are ready for commercial consumption. Commercial companies don’t talk about a new technology until it’s close to beta. What was being talked about a year before were now a reality, but the conversation is still about how interesting and amazing things would be next year.

  • Search, Security + Governance

Search was the biggest theme and was starting to be done well, Adrian said. As there is talk about the Data Lake, “the undifferentiated pile” that data is pulled out of out, search is becoming critical. The rise of interactive analytics on Hadoop was the second big theme, but there was “a lot of misstatement of it being real time.”

John Furrier asked if applications were the big thing for Hadoop, or is it still stuck on analytics. Adrian said that “we are in the early moments of the platform being accepted as a vehicle to moving forward.” The main gaps are security and governance. “We have to have confidence about the security of the platform, that’s the number one issue,” Adrian says.  The market wants that before they can rely on it. “We need a governant platform.” Knox, he said, is addressing the security aspect up to a point. Once those gaps are filled, the next step would be an explosion of apps.

Securing standards

 

The biggest issue for CEOs uncovered by Gartner’s research comes down to IT was trust. Where data is concerned  aspects such as who owns it, provenance, security are still very unclear. Gartner customers are mainstream corporations where trust is very important. 31 percent of those stated they have no plans to initiate Big Data investments over the next two years in a survey targeting IT departments across all industries and geographies. Adrian explained that, in his opinion, the main reason was they didn’t have a real understanding of the value. Many are still asking if Hadoop is just hype, or if it brings real value.

When a technology’s value is proved, it becomes hardened. However, while at the Hadoop Summit, people are already talking about new technologies, such as Spark. “There’s always consolidation in the middle,” Adrian said, referring to the current state of Hadoop, while more innovation is hitting the market.

Trends exciting the cloud market

 

Asked about Amazon’s role in the cloud and Big Data markets, Adrian said that “web native organizations, businesses that started on the web, it is unlikely that they will go to an on-premise world. If they started in the cloud, they will stay in the cloud.”

When it comes to Amazon, it is hard to have a clear view, as they are pretty opaque. Amazon is a great place to experiment, people are doing interesting things in the cloud, but in the case of Amazon poster children, there needs to be some insight about how much of their portfolio is really in the cloud. “Amazon’s stated vision is goal is high volume, low margin provider. Most of us think about extracting margin. Amazon is determined to drive margin down, it’s really hard to compete with them,” he added

“People don’t want black boxes, unless they are sure the black box will work seamlessly, without errors,” Adrian stated.

  • Who’s the Red Hat of Hadoop?

Asked if a Red Had of Hadoop might appear, Adrian said that there wouldn’t be, “not in the next three years.”

“At every layer of the stack, there are multiple alternatives,” Adrian said. Everything is substitutable, some of the options being open source, some not. “Every distribution is a composition and there’s lots of room for more compositions. I don’t think we’re anywhere near the end of the innovation cycle.”

  • More trends to watch

Asked what he would keep an eye in the future, Adrian said he would focus on the gaps he had mentioned – security, governance, and administration. As there was more information shared in those aspects, he would need to research them more, in order to be able to advise Gartner customers about investing or not in Big Data.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU