UPDATED 14:42 EST / MARCH 01 2016


Cloudera launches open-source effort to harness Hadoop for security analytics

Hadoop will soon be brought up much more often in enterprise security discussions if Cloudera Inc.’s newest community effort achieves its goal. The Open Network Insight (ONI) project launched on GitHub this week to help organizations take advantage of data crunching platform’s processing power in their breach prevention efforts.

The documentation for the tool explains that it’s a mix of free technologies from the Hadoop ecosystem and machine learning algorithms that Cloudera created in collaboration with a number of leading network protection vendors. Once all of its components are properly deployed, ONI starts pulling traffic logs from the environment that it’s protecting into the Hadoop File System, where they’re analyzed in stages.

First, the data is run through Cloudera’s algorithms using Apache Spark to filter out the mundane communications that account for the bulk of the activity in the average organization’s network. Then, the remaining logs are correlated with information from other systems in the environment, like the firewall that controls who can access what application, to produce a full picture of the situation at hand. The results are analyzed once again in the final stage of the process to single out suspicious traffic, which is then displayed through ONI’s graphical interface.

From there, the data can be manually explored using built-in visualization capabilities to determine for certain that a breach has occurred. According to Cloudera, the software will eventually also be able to check employee-owned devices for signs of foul play, which should broaden its appeal considerably. Users won’t have to wait on the vendor to slowly add the functionality they require, however. The fact that the source-code of ONI is freely available means that every organization is able to customize the project according to its specific security demands.

Image via Geralt

A message from John Furrier, co-founder of SiliconANGLE:

Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.

Join Our Community 

Click here to join the free and open Startup Showcase event.

“TheCUBE is part of re:Invent, you know, you guys really are a part of the event and we really appreciate your coming here and I know people appreciate the content you create as well” – Andy Jassy

We really want to hear from you, and we’re looking forward to seeing you at the event and in theCUBE Club.

Click here to join the free and open Startup Showcase event.