UPDATED 09:19 EDT / NOVEMBER 09 2011

Hadoop World Day 1: A Booming Big Data Ecosystem

Hadoop World’s first day started with a lot of buzz as the Hadoop ecosystem flourishes.  Cloudera CEO Mike Olson kicked off the conference with a brief history lesson, and how companies are now embracing Hadoop.  He cited some vendors building applications for Hadoop such as WibiData, Cetas, Revolutions Analytics, Digital Reasoning, and Tidemark.

Accel Partners then announced a $100 million fund for Big Data.  The funding shows just how important Hadoop is in solving real-world problems brought about by consumerization, the data analytics ecosystem and making Big Data tools available for everyone to use and not just for Data Scientists.

Hearing directly from the community, Hadoop COO Kirk Dunn and NetApp Senior Director of Data Center Solutions Jeff O’Neal appeared on theCube with SiliconAngle founder and theCube co-host John Furrier.  Dunn pointed out that Hadoop isn’t good for relational data but for recommendation engines.  O’Neal talked about their new solution, NetApp Open Solution for Hadoop, which was brought to fruition in a partnership with Cloudera. Facebook engineer Jonathan Gray also made an appearance at Hadoop World and discussed their use of Hadoop HBase, a growing platform for enabling realtime operations and ananlytics.  HBase includes sorted and column-oriented data organization, high write throughput (a big plus for HBase), horizontal scalability and auto-failover–just some of the reasons Facebook took a liking to Hadoop.

Hadoop CTO Amr Awadallah also appeared on theCube to talk about how Hadoop is similar to Linux in terms of how they both provide a file system and a platform for running applications.  But what differentiates Hadoop is that it is designed to run on clusters while Linux isn’t.

“Right now the fact that you can buy a single server with 8 to 16 cores with a few terabytes hard drive means that it’s more likely for new hardware to disrupt that,” said Awadallah about how easy it is to set up and prepare an analysis-based server set. “I’m seeing cases where companies say this Hadoop thing is so cool, what can I do with this? And then there’s companies with a problem and no idea Hadoop exists, so when they discover it they have a readymade solution.”

During the interview, the topic veered into online gaming, in which Awadallah proudly announced that he is a gaming geek and every Thursday they play Modern Warfare 1.  It turns out the Cloudera guys are challenging competitors!

Splunk, a big data aggregation and intelligence tool, announced that it would launch a special edition for Hadoop dubbed as Splunk Enterprise.  The solution integrates the analytics engine into Splunk in order to provider a number of key advantages to users that leverage both technologies.

Another guest on theCube was Daniel Abadi, a Yale University assistant professor and co-founder and chief scientist at startup Hadapt.  While still a grad student at MIT, Abdi developed three data systems that were used by researchers or industry before they were commercialized.  Back then he had no idea that it would become a multi-billion dollar industry.  He is now part of a group at Yale that that runs a large data systems lab.  Another project he plans on pursuing is creating a graphic database system.

Watch the live coverage of Hadoop World 2011 Day 2 via #TheCUBE at siliconangle.tv.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU