Hadoop will soon be brought up much more often in enterprise security discussions if Cloudera Inc.’s newest community effort achieves its goal. The Open Network Insight (ONI) project launched on GitHub this week to help organizations take advantage of data crunching platform’s processing power in their breach prevention efforts.

The documentation for the tool explains that it’s a mix of free technologies from the Hadoop ecosystem and machine learning algorithms that Cloudera created in collaboration with a number of leading network protection vendors. Once all of its components are properly deployed, ONI starts pulling traffic logs from the environment that it’s protecting into the Hadoop File System, where they’re analyzed in stages.

First, the data is run through Cloudera’s algorithms using Apache Spark to filter out the mundane communications that account for the bulk of the activity in the average organization’s network. Then, the remaining logs are correlated with information from other systems in the environment, like the firewall that controls who can access what application, to produce a full picture of the situation at hand. The results are analyzed once again in the final stage of the process to single out suspicious traffic, which is then displayed through ONI’s graphical interface.

From there, the data can be manually explored using built-in visualization capabilities to determine for certain that a breach has occurred. According to Cloudera, the software will eventually also be able to check employee-owned devices for signs of foul play, which should broaden its appeal considerably. Users won’t have to wait on the vendor to slowly add the functionality they require, however. The fact that the source-code of ONI is freely available means that every organization is able to customize the project according to its specific security demands.

Image via Geralt