Big Data Security and Intelligence into 2013


The Information Age is quickly becoming the age of information-overload–to combat this, we’ve seen search, curation, and even intelligent agents start to take shape out of the desire to make sense of all the data available. For big business it’s become a whole different problem: the complexity of data that flows through their systems means numerous new ways that attackers can exploit loopholes or amplify flaws in operations. In the current culture of cybersecurity this could lead to leaking internal and secret customer data, losing millions of dollars to fraud, or worse.

To combat this, Big Data solutions have been leveraged across numerous different enterprises to help prevent or discover fraud and shore up potential security holes.

Wikibon’s Jeff Kelly recently released his Big Data market revenue report and in that many different outfits can be seen pouring hard earned profits into this new technology. Many of them have their own projects that reflect the use of Big Data for security.

IBM Security Intelligence


IBM hit the top of the list for Big Data revenue and they’re offering a massive real-time forensic computing tool that leverages big data analytics (including ad hoc queries and visualization) alongside data correlation and anomaly detection for real-time processing with rules and reporting for security operations.

One big thing about the IBM solution that fits nicely into Kelly’s market report is the increased adoption of Hadoop for Big Data products. IBM’s product roundly and proudly trumpets their use of Hadoop-based analytics as part of its core product set.



For log analysis and maintenance and intrusion detection, Splunk has been at the forefront of everything analytics for a very long time. I’m personally familiar with how Splunk has been used in the past for cybersecurity and how their approach to Big Data analysis–and specifically unstructured data from a multitude of otherwise unconnected sources–could be used to protect critical infrastructure.

Splunk has a deep relationship with security and their primary products that address that lay around enterprise security, advanced persistent threats, and log analysis.

Their presence in the market list isn’t at all surprising, especially how Splunk describes the necessity of Big Data analysis for advanced persistent threats (APT.) Many attackers take advantage of holes left in the infrastructure of culture of enterprise to sneak in and do their damage–but they don’t often just do it once: they often leave behind malware or backdoors as we’ve seen in the recent hacks against US journalism sites such as the Wall Street Journal and the New York Times.

Big Data is an important part of watching for patterns that reveal the activity of malicious actors inside a network. Sometimes this means taking data not just from what computers are doing, but also what people are doing, or how information is moving to and from networks. This means huge amounts of unstructured data need to be processed, context sifted through, and rules designed to identify patterns that are out-of-the-norm.

Others in the Big Data security sphere


Also in the list, EMC pops out as a big name in storage and with their acquisition of RSA adds themselves nicely to Big Data security as well. The company released an excellently concise white paper on the subject of how big data can transform security–by putting together infrastructure, analytics, and intelligence (the big three of every security management perspective.)

Cisco recently acquired Cognitive Security, a Czech network security startup, to develop a big data solution to detecting anomalies. As a networking outfit, who primarily sell and build networks, being able to collect data from distant nodes and analyze the information to determine if anything out of the ordinary happened to be going on.

On a level of schadenfreude, it’s amusing to see Booz Allen Hamilton in the market list: after all, they had one of their servers hacked by Anonymous in 2011 with 9,000 internal e-mails leaked.

Overall, it’s obvious that 2013 outfits that work with Big Data are continuing to see how effective it is in supplementing standard security practices—primarily because analysis of threats both spontaneous and persistent requires real-time analysis of both current and historical data. It also shows that much of the market is angling itself to prepare for more businesses seeking solutions that will give them not just intrusion detection but swift answers on how to deal with intruders once discovered and Big Data is leading that charge.