UPDATED 09:19 EDT / NOVEMBER 09 2011

Hadoop World Day 1: A Booming Big Data Ecosystem

Hadoop World’s first day started with a lot of buzz as the Hadoop ecosystem flourishes.  Cloudera CEO Mike Olson kicked off the conference with a brief history lesson, and how companies are now embracing Hadoop.  He cited some vendors building applications for Hadoop such as WibiData, Cetas, Revolutions Analytics, Digital Reasoning, and Tidemark.

Accel Partners then announced a $100 million fund for Big Data.  The funding shows just how important Hadoop is in solving real-world problems brought about by consumerization, the data analytics ecosystem and making Big Data tools available for everyone to use and not just for Data Scientists.

Hearing directly from the community, Hadoop COO Kirk Dunn and NetApp Senior Director of Data Center Solutions Jeff O’Neal appeared on theCube with SiliconAngle founder and theCube co-host John Furrier.  Dunn pointed out that Hadoop isn’t good for relational data but for recommendation engines.  O’Neal talked about their new solution, NetApp Open Solution for Hadoop, which was brought to fruition in a partnership with Cloudera. Facebook engineer Jonathan Gray also made an appearance at Hadoop World and discussed their use of Hadoop HBase, a growing platform for enabling realtime operations and ananlytics.  HBase includes sorted and column-oriented data organization, high write throughput (a big plus for HBase), horizontal scalability and auto-failover–just some of the reasons Facebook took a liking to Hadoop.

Hadoop CTO Amr Awadallah also appeared on theCube to talk about how Hadoop is similar to Linux in terms of how they both provide a file system and a platform for running applications.  But what differentiates Hadoop is that it is designed to run on clusters while Linux isn’t.

“Right now the fact that you can buy a single server with 8 to 16 cores with a few terabytes hard drive means that it’s more likely for new hardware to disrupt that,” said Awadallah about how easy it is to set up and prepare an analysis-based server set. “I’m seeing cases where companies say this Hadoop thing is so cool, what can I do with this? And then there’s companies with a problem and no idea Hadoop exists, so when they discover it they have a readymade solution.”

During the interview, the topic veered into online gaming, in which Awadallah proudly announced that he is a gaming geek and every Thursday they play Modern Warfare 1.  It turns out the Cloudera guys are challenging competitors!

Splunk, a big data aggregation and intelligence tool, announced that it would launch a special edition for Hadoop dubbed as Splunk Enterprise.  The solution integrates the analytics engine into Splunk in order to provider a number of key advantages to users that leverage both technologies.

Another guest on theCube was Daniel Abadi, a Yale University assistant professor and co-founder and chief scientist at startup Hadapt.  While still a grad student at MIT, Abdi developed three data systems that were used by researchers or industry before they were commercialized.  Back then he had no idea that it would become a multi-billion dollar industry.  He is now part of a group at Yale that that runs a large data systems lab.  Another project he plans on pursuing is creating a graphic database system.

Watch the live coverage of Hadoop World 2011 Day 2 via #TheCUBE at siliconangle.tv.


A message from John Furrier, co-founder of SiliconANGLE:

Support our open free content by sharing and engaging with our content and community.

Join theCUBE Alumni Trust Network

Where Technology Leaders Connect, Share Intelligence & Create Opportunities

11.4k+  
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+ 
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.

SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.