In part two of the Wikibon big data manifesto, Jeff Kelly compares traditional analytics and the tools associated with it with big data, and the need for new tools to effectively process the growing quantity of unstructured data generated within the enterprise.

Kelly looked into the mainstream way of ingesting data first. Companies would generate data, mostly structured data, which is then handled by integration tools that attempt to model it into rows and tables in a spreadsheet. This information is then loaded into a data warehouse (where data volumes remain relatively small) and users can access it either via visualization tools, e.g. dashboards, or BI tools such as SAP BusinessObjects.

This approach, however, cannot be sustained today, when big data outpaces the capacity of traditional tools. Kelly attributes several key characteristic attributes to this increasingly popular catchphrase, namely: the volume of the data and the great pace in which it’s being created, which in turn introduced a need for real-time analytics. He pointed out that the variety or number of formats in which this big data is available also increased.

Kelly continued to highlight the main sources of big data today, including sensors and hardware, mobile, e-commerce transactions and of course social media interactions, one of the titans of this list.

In short, big data translates into new requirements: new tools that can ingest large amount of unstructured data from a great number of sources, while being cost-efficient and being able to maintain near real-time performance.

Companies like ClickFox are taking different approaches to addressing this issue in a number of areas. The startup focuses on customer service in particular, and helps companies create an outline of customer experience rather than simply gauge market conditions based on more shallow interactions.