Social media gathers people together into communities and as people interact they produce and spin off tremendous amounts of data. That data is so considerable and detailed that they only way to approach it is through the praxis of big data theories and algorithms. Sociologist Marc Smith (@marc_smith on Twitter) with the Social Media Research Foundation is at the forefront of producing visualization techniques.

John Furrier of SiliconANGLE and Dave Vellante of Wikibon.org brought him into TheCube at StrataConf 2012 to speak about his insights about big data and its intersection with social media. Smith believes that we’re on the revolutionary cusp necessary to start constructing the tools to engage scholarship about big data and social data.

When Furrier asked Smith how he felt that big data has been changing our understanding of society and social media, he felt that it’s the next big revelation that will break open our intuition of the “big picture.”

There are a lot of patterns that were difficult to discover because the data wasn’t there and we’re going to discover those patterns. Smith says, Whoever “races to the top of the Big Data mountain first will see that vista” and they’ll be the ones who will be able to exploit those patterns first. What Marc believes is that big data will give social scientists the ability to see society.

As in the ability to see society in a “satellite view” that we haven’t ever had before.

The graph represents a network of up to 1000 Twitter users whose recent tweets contained “bigdata”. The network was obtained onTuesday, 28 February 2012at01:55 UTC. There is an edge for each follows relationship. There is an edge for each “replies-to” relationship in a recent tweet. There is an edge for each “mentions” relationship in a recent tweet. There is a self-loop edge for each tweet that is not a “replies-to” or “mentions”.

Twitter itself provides an amazing snapshot within the ocean of publicly available social data. Because every comment and connection is trapped by Twitter and can be captured by someone watching means that we can map the relationships between different players and even visualize conversations on specific subjects. For the study of sociology, this is an amazing breakthrough in order to see the flow of ideas, and how different cliques interact.

As Smith says, the scholarship could benefit greatly from tools capable of sieving out patterns from these interactions allowing sociologists (or marketers, enterprise PR, etc.) to transform these big data models to do work for the people who want to know. The enterprise space has been looking into how to use this to outperform their competitors and better serve their customers—but as Smith mentions, scholarship in this realm will move us forward in understanding how we can use big data to serve the entire community.

The project imaged above, from NodeXLGraphGallery.org, uses an open-source template for graphing network data that works with Microsoft Excel 2007 and 2010. It uses a Microsoft Public License (Ms-PL) which allows for derivative works based on the template meaning that it can be spread into academic research easily.

We’ve seen before that visualization is extremely important when it comes to making big data human readable and for distilling trends from information. Social media is inherently a human process with a multitude of communities, people, and behavior all interacting in very complex ways.

With a product like NodeXL and Social Media Foundation delivering ideas on how to better approach products that view these networks will give a powerful starting point to bring big data into scholarly pursuits.