Big data has presented us with some pretty unique challenges. Storage and analysis are the ones we hear about most frequently, but equally important is the need to tackle problems related to the security and privacy of the data that companies collect.

To address these needs, Cloud Security Alliance and Fujitsu Labs, two of the world’s biggest data vendors, have gotten together to form the Big Data Working Group, in the hope that they can encourage more collaboration among data vendors and come up with permanent solutions to the security challenges they face.

The move comes at a time when companies can’t get enough of big data, which has the potential to provide insights worth millions of dollars so long as those companies know how to exploit it. But while no one denies the wonderful opportunities that big data possesses, there are also plenty of risks – so many in fact, that if data is not handled with care and expertise, anyone attempting to juggle with it could easily find themselves in a whole heap of trouble.

Big data issues

A recent article in Information Week highlights the numerous concerns that come with handling big data, concerns that both companies and vendors alike will soon have to get a grip on. Admittedly, these security issues are nothing new, but the need to address them has never been more pressing as they become magnified by the sheer volume, variety and velocity of all that big data being generated.

The most frequently discussed problem is that of data theft. With the number of online interactions increasing exponentially each year, hackers have never had so many incentives to go after all of that oh-so important financial data.

Failing to protect their customer’s personal information from hackers could cripple a company’s reputation, but this isn’t the only worry. Other problems include cloud storage, employee access, the vulnerability of interconnected supply chains that rely on data, and finally, privacy concerns. All of them have the potential to spell disaster for companies that don’t get their policies right.

A coordinated response

These problems exist because of big data, but the twist comes in the fact that big data itself could well provide the answer to many of these security issues.

Sreeranga Rajan, director of software systems innovation at Fujitsu Laboratories of America, explains:

“By collaborating as a global community of thought leaders and researchers, we are not only looking to help the industry overcome these challenges but also to leverage new opportunities for the monitoring and detection of security threats enabled by big data.”

Accordingly, the group plans to use its big data know-how to first of all identify the biggest threats to the industry, and secondly, to establish industry-wide best practices for security and privacy that can be adopted by organization and governments across the world.

To do so, one of the first tasks the Big Data Working Group has set itself is to research and later publish reports on six themes that it has identified as being fundamental to the future of big data’s integrity: data-scale crypto; cloud infrastructure; data analytics for security; framework and taxonomy; policy and governance; and privacy.

The group says that the first of these reports should be ready by the autumn, and will outline the fundamentally different technical and organizational problems faced when addressing big data security. Later reports will focus on the creation of best practices for big data security and privacy, and the establishment of security and privacy test beds, which will be needed to strengthen the security and privacy of cloud platforms.

In the meantime, the group is calling on other interested players to join them in their common cause, a call that has so far been heeded by eBay and Verizon. Once the word spreads, hopefully more organizations will hop on board and help the group to meet its goals

After all, it’s in everybody’s interests that the group succeeds in its aims. Their success will be beneficial not just for data vendors, but most importantly, for the people whom the data they manage actually belongs too  – each and every one of us.