With all the conversations around the NSA and Prism monitoring our data it’s important to learn what it takes to mine the massive amounts of data.

The Atlantic had a great post on the challenges of technology advancements and how hard it is to apply the policy of getting data on bad actors.    To me the core issues are: 1) tech policy isn’t that mature and 2) yes we do need to keep an eye on what is acceptable and have transparency.

The challenge is who do you trust?

The courts have failed to develop a robust system for applying the Fourth Amendment meaningfully to the questions of the 21st century.  Comments on my Facebook page are going strong mostly saying that this is not a big issue as it relates to what is already happening to our privacy.

Douglas Karr a friend of mine on Facebook had an interesting quote “I think it’s amazing that people trust the government with their data given the IRS releasing data to political opposition forces AND a “TOP SECRET” government program getting released. If you trust the government with this data, I question your intellect.”

Frank Paynter on Facebook also commented “There’s nothing new here. The Hepting/AT&T case back in 2006 established the parameters for secrecy and corporate cooperation with the NSA under the rules of the FISA court. Rather than explain that, some corporate executives have chosen the disingenuous path of suggesting that their data is not available to the government except in limited circumstances. It makes them look dishonest and foolish. To say there is nothing bad happening here is–in my opinion–to say that the USA PATRIOT Act is not “bad.” There are two schools of thought on that. Many of us who would like to see that act revised and executive powers restricted think that there is indeed something bad happening here and a full, open, and honest discussion of how the government extracts personal data from corporations is past due.”

Peter Keates weights in on the obvious that our gov’t already monitors us. “NSA for example is already able to read data on the web, without any permission from Google etc.. it’s not a secret… so Google opening user data to government is not the problem, government already read your data…”

Here is a talk from ex NSA entrepreneurs from sqrrl a big data startup where the founders where from the NSA. How to scale big data tech.

Adam Fuchs, CTO and Co-Founder of sqrrl, does a whiteboard session with Wikibon’s cofounder and industry analyst Dave Vellante on best practices in Big Data application development used by the NSA and hyperscale web companies.