A little known company called Pepperdata Inc. that specializes in managing and fine-tuning Hadoop clusters to deliver better performance, has just grabbed $15 million in a series B round to fire up its Big Data platform.

New investor Wing Venture Capital led the round, joined by existing investors, including Signia Venture Partners and Yahoo Chairman Maynard Webb. New strategic investors Citi Ventures and Silicon Valley Data Capital also invested, bringing its total investment to $20 million, according to TechCrunch.

Pepperdata claims to power some of the world’s largest Hadoop environments across industries including the consumer electronics, energy, media, finance and telecommunications sectors. The company claims its customers see significantly increased reliability and capacity within their existing Hadoop deployments, but how exactly does it do that?

The answer, in a nutshell, is that Pepperdata makes Hadoop smarter by optimizing cluster performance. And Pepperdata should be pretty well qualified to do that, for its co-founder Sean Suchter previously happened to be on Yahoo! Inc’s web search engineering team, which is of course where Hadoop also began its life.

The problem with Hadoop is that, despite being the most popular tool for sorting through masses of Big Data, it’s not that great when it comes to scheduling tasks, according to Pepperdata’s other co-founder Chad Carson. Hadoop is often used by numerous departments within an organization, and it means that lower-priority workloads may often be given precendence over higher-priority workloads, which is inefficient to say the least.

That’s because Hadoop in its native form lacks a mechanism to prioritize jobs based on needs or status, but luckily, this is what Pepperdata provides. It gives users a way to fine-tune Hadoop’s job scheduler, giving resources to the jobs that need them most urgently, while ensuring organizations can stick to the terms of their service level agreements (SLAs).

“We’ll look at the jobs and see that a low priority guy is flooding the network or taking [hard] disk, and reach into the interfering job and slow it down just enough to give [the higher priority one access],” Suchter said.

Pepperdata can do so because it’s smart software, Carson told TechCrunch. It’s able to analyze capacity usage throughout the system and manage it more effectively. For example, some jobs might use more memory than is actually required – Pepperdata is able to free up this ‘unnecessary’ memory and make it available to alternative jobs in the queue, thus helping the system run much more efficiently. Indeed, Pepperdata claims its customers see a 30 to 50 percent throughput gain on average.

With its new funding, Sunnyvale-based Pepperdata is planning to “expand its global reach and the presence of its sales and marketing operations”, which will entail increasing its workforce from around 30 employees now to about 50 by the end of the year.

Photo Credit: chefranden via Compfight cc