So far we have been hearing and discussing about Big Data only, but the next big thing is sorting it all out. Data is everywhere and always has been, but now we’re able to collect, store and process it all.  It’s in your smartphone, your car, home, tablet, your digital camera, and even GPS devices.  And the data we keep is progressively multiplying, making it hard to manage in an effective way.  Here’s where data sorting companies come into play. One of the startups that that offer similar services is Splunk, a San Francisco-based start-up whose software indexes huge quantities of machine-generated data into searchable links. Companies search those links in the same way as a person searches Google to analyze customer behavior in real time.

Splunk was founded in 2004, much earlier than the time when the actual term ‘Big Data’ became popular.  They unveiled the first version of its software in 2005.  With around 3,200 customers in more than 75 countries, Splunk is now trying to establish itself in a territory long controlled by giant business-technology vendors like Oracle and I.B.M.  We heard firsthand their goals for big data, Hadoop and client acquisition during Hadoopworld earlier this year.

Some of the prominent customers of Splunk include Zynga that uses its software monitor game function to determine where players get stuck or quit playing. This, in turn, allows Zynga to tweak games in real time to retain players.

Other companies using Splunk’s services include Macy, Edmunds, a Japanese Tsunami relief organization, and hundreds of governmental agencies.

“For a while there, people felt like everything that needed to be solved had been solved and that big companies would inevitably find all of the white space in enterprise,” said David Hornik, an investor at August Capital, which invested $3 million in Splunk in 2004.

“Splunk is really the poster child for thinking differently about an enterprise challenge and creating a platform that ends up really being disruptive and valuable.”

The start-up got a total of $40 million in venture capital at that time from August Capital, Ignition Partners, JK&B Capital and Sevin Rosen Funds.

Another company to look at is Attivio, which has recently secured a U.S. patent (No. 8073840) for its unique inverted index, the secret sauce behind the company’s Unified Information Access platform that allows it to access, join and query all data types. The AIE platform allows users and developers to access and join both structured and unstructured data dispersed among previously silo’ed databases and systems while maintain the relationships between data elements.

Taking a look at other developments in Big Data, we have our own expected trends for 2012.  Check out the complete coverage.