Enterprise can derive useful insights from Big Data but this advantage comes at a cost, with large datasets rapidly becoming tempting targets for hackers and other cybercriminals. To counter this threat, IBM has just unleashed its latest security solution, Security Intelligence with Big Data, which aims to combine analytics with intelligence to head off these unwanted threats.

IBM’s Security Intelligence with Big Data has been designed to help enterprises tackle the most pressing security challenges they face, such as insider threats, fraud and persistent threats. The solution unites the anomaly detection and real-time security capabilities of IBM QRadar Security Intelligence Platform with the custom analysis and data exploration assets of IBM InfoSphere BigInsights, giving enterprises new perspectives on the threats they face.

The idea is that by combining custom analytics and real-time correlation across both structured and unstructured data sets, enterprises will be able to protect themselves by widening their investigations to new data types. Security Intelligence with Big Data will allow organizations to find answers to questions they never even though to ask, analyzing structured security data alongside their unstructured enterprise data to locate malicious threats hidden within their mass of Big Data.

Real-time correlation and anomaly detection within diverse network and security are being touted as the key capabilities of Security Intelligence with Big Data, but these are just one aspect of the solution. Other capabilities include flexible Big Data analytics, high-speed querying of security intelligence, forensics for deeper visibility in network activity, and a graphical data visualization tool for exploring Big Data.

IBM’s Security Intelligence with Big Data has already been deployed by a number of large corporations, including the Depository Trust & Clearing Corporation (DTCC), which is using the system to gain more real-time awareness about its security threats.

“We need to move from a world where we ‘farm’ security data and alerts with various prevention and detection tools to a situation where we actively ‘hunt’ for cyber-attackers in our networks,” said DTCC’s chief information security office Mark Clancy.

“IBM’s Security Intelligence with Big Data solution gives us a practical way to gain visibility across our environment. We’re gaining real-time security awareness and meaningful insight into historical activity across years of diverse data.”

IBM isn’t the only provider to pinpoint Big Data as the most fundamental tool for detecting security threats in the future. EMC’s RSA Security division recently gave details of its own Security Analytics tool designed to harness Big Data analytics to help companies defend their digital assets against today’s most sophisticated internal and external threats.

The research firm Gartner is equally convinced that Big Data is the only solution for information security problems. Analyst Neil MacDonald noted that this solution might require a combination of firewall logs, reputational analysis, network packet data, and other contextual information in order to determine if an attack has taken place – something that several larger banks and organizations have already attempted to do by building their own Big Data security tools. However, he points out that solutions like IBM’s Security Intelligence with Big Data are likely to prove more cost-effective.

MacDonald adds that this is still an emerging technology, but eventually it could become one that can even be applied to small and medium-sized enterprises.