All Data Is Relevant Now
The Wall Street Journal posted a story this week about the NSA data mining program and the legal framework that supports it. The story zeroed in on what it characterized as the evolving interpretation of one word – ‘relevant’ – in the Patriot Act and how it enables the NSA and other government agencies to legally collect and mine vast troves of data.
Since 9/11, the Patriot Act has required the NSA to petition the Foreign Intelligence Surveillance Court whenever it wants to collect and analyze otherwise private data, such as mobile phone or email records, as part of an investigation. Since the mid-2000’s, the court requires the NSA to show the data in question is ‘relevant’ to the investigation and likely to result in information that would aid investigators.
Here’s where things get interesting. The Wall Street Journal story maintains that the interpretation of the term ‘relevant’ has expanded to the point that it is essentially a meaningless term. I agree that, from a Big Data perspective, the term ‘relevant’ is no longer, well, relevant. This isn’t because the court is using a broader interpretation of the word, however, but because technology has advanced to the point where, for example, petabytes of data associated with millions of Americans’ mobile phone usage actually are relevant to anti-terrorism investigations focused on specific individuals.
Consider that ten plus years ago, investigators wouldn’t even have thought to collect such vast amounts of data. It wasn’t relevant then because there was no way to effectively store it or analyze it. But today such data is relevant thanks to technologies like Hadoop and Accumulo, and Data Science has illustrated that analysis of seemingly unrelated data sources can in fact reveal impactful insights.
The NSA program and the specific algorithms in use aren’t public knowledge, so I can’t speak to precisely how all this data is being integrated and interpreted. But there are examples in the commercial world of value hidden in unlikely data sources. Speaking on theCUBE recently, Splunk’s Sanjay Mehta recounted the story of a Japanese real estate company that is collecting and analyzing operational data created by elevators because such data is a predictor of lease renewals, for example.
All of this is not to say that the NSA should be collecting and analyzing all of our mobile phone records. Where we draw the line in terms of balancing privacy and security is a subjective judgment that we as a society must make. But the fact is, Big Data technology makes virtually all data ‘relevant’, or at least potentially valuable, if we decide to make use of it.
A message from John Furrier, co-founder of SiliconANGLE:
Your vote of support is important to us and it helps us keep the content FREE.
One click below supports our mission to provide free, deep, and relevant content.
Join our community on YouTube
Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.
THANK YOU