UPDATED 07:15 EDT / APRIL 17 2015

Pepperdata grabs $15M to help make Hadoop act smarter

2067875072_3a38d2d404A little known company called Pepperdata Inc. that specializes in managing and fine-tuning Hadoop clusters to deliver better performance, has just grabbed $15 million in a series B round to fire up its Big Data platform.

New investor Wing Venture Capital led the round, joined by existing investors, including Signia Venture Partners and Yahoo Chairman Maynard Webb. New strategic investors Citi Ventures and Silicon Valley Data Capital also invested, bringing its total investment to $20 million, according to TechCrunch.

Pepperdata claims to power some of the world’s largest Hadoop environments across industries including the consumer electronics, energy, media, finance and telecommunications sectors. The company claims its customers see significantly increased reliability and capacity within their existing Hadoop deployments, but how exactly does it do that?

The answer, in a nutshell, is that Pepperdata makes Hadoop smarter by optimizing cluster performance. And Pepperdata should be pretty well qualified to do that, for its co-founder Sean Suchter previously happened to be on Yahoo! Inc’s web search engineering team, which is of course where Hadoop also began its life.

The problem with Hadoop is that, despite being the most popular tool for sorting through masses of Big Data, it’s not that great when it comes to scheduling tasks, according to Pepperdata’s other co-founder Chad Carson. Hadoop is often used by numerous departments within an organization, and it means that lower-priority workloads may often be given precendence over higher-priority workloads, which is inefficient to say the least.

That’s because Hadoop in its native form lacks a mechanism to prioritize jobs based on needs or status, but luckily, this is what Pepperdata provides. It gives users a way to fine-tune Hadoop’s job scheduler, giving resources to the jobs that need them most urgently, while ensuring organizations can stick to the terms of their service level agreements (SLAs).

“We’ll look at the jobs and see that a low priority guy is flooding the network or taking [hard] disk, and reach into the interfering job and slow it down just enough to give [the higher priority one access],” Suchter said.

Pepperdata can do so because it’s smart software, Carson told TechCrunch. It’s able to analyze capacity usage throughout the system and manage it more effectively. For example, some jobs might use more memory than is actually required – Pepperdata is able to free up this ‘unnecessary’ memory and make it available to alternative jobs in the queue, thus helping the system run much more efficiently. Indeed, Pepperdata claims its customers see a 30 to 50 percent throughput gain on average.

With its new funding, Sunnyvale-based Pepperdata is planning to “expand its global reach and the presence of its sales and marketing operations”, which will entail increasing its workforce from around 30 employees now to about 50 by the end of the year.

Photo Credit: chefranden via Compfight cc

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU