As the Big Data technology megatrend steams ahead, we’re seeing the rapid development of Applied Big Data. Applied Big Data refers to technology platforms that use Big Data computing techniques, platforms, and tools to solve problems that would be unsolvable without massive computing power applied to massive quantities of data. Applied Big Data is differentiated from general building-block components like Hadoop. Rather, the Applied Big Data space is filled with task-specific applications aimed at concrete and highly valuable goals.

In particular we’ve seen strong recent focus on Applied Big Data for the physical world.  The extreme complexity of many physical phenomena has made offline problems a natural for Big Data. To some degree this idea is nothing new. Highly visible science projects like genome mapping, measurement of global climate change, or searching for Earth-like planets all depend on the ability to process very large data sets. What is new is the widespread sense that these same ideas can benefit lots of the pragmatic and even mundane mechanisms that keep the wheels of society turning.

Applied Big Data for Brick-and-Mortar Retail

.

A great example is Applied Big Data for brick-and-mortar retail, in which we seek to draw actionable inferences from customers’ behavior in brick-and-mortar retail environments. To do so, we must account for diverse factors like the physical location of shoppers and employees in the store (and be able to differentiate between the two); the layout, fixtures, and planogram of the store; staffing schedules; complete detail on actual sales, and even the weather. Input sources can include video cameras, Wi-Fi tracking tags, RFID, and other in-store systems like those for Point-of-Sale (POS), staffing, and task management.

Via these data sources, retailers presently using our system are collecting about 10,000 data points per store visitor, and we expect that figure to go up over time as more and richer data sources become available. Across our full customer set of more than sixty retail chains the RetailNext system collects roughly 57 petabytes (57,000,000 GB) of raw data across more than 300 million shopping trips per year. We process this flood of raw data into the trillions of analytical data points (one level more abstracted than the raw data) that then drive the measurement, analysis, and direct management for which retailers use our system.

The potential for retailers is as large as the data sets. Retailers can gain a precise, factual understanding of how shoppers move around their stores – where they go, in what order, how long they stay, when they come to the store, and how all of these questions map to actual sales. Retailers have optimized store layouts, fixtures, staffing, and even product offerings based on what they learned. CPG manufacturers also use this category to more thoroughly understand how their packaging, merchandizing, and marketing decisions affect the full path to purchase.

The upside for these companies is breathtaking. By taking an Applied Big Data approach to their brick-and-mortar locations, retailers have seen results like these:

• American Apparel increased same-store sales between 30% and 40% and reduced theft 16%.

• Montblanc increased same-store sales 20%.

• Brookstone cut theft approximately one million dollars a year.

• Family Dollar remodeled more than 1300 locations in the first nine months to apply optimizations discovered by measuring and analyzing shopper behavior.

Improvements like these are possible because this environment is one that historically has lacked all but the crudest of measurements. Blunt error-prone techniques like surveys, store visits, basic traffic counting, and context-free shopping basket analysis have taken brick-and-mortar retailers only a small way toward the optimized store experience. In fact, McKinsey and Company estimates that Big Data analytics can improve retailers’ operating margin more than 60%. That’s a pretty powerful weapon in a highly competitive space like retail.

Such improvements have been available to online retailers for more than a decade, leaving online retail as our best indicator of the new category’s upcoming adoption curve. Google Analytics, for instance, is in use on exactly 50% of the Alexa One Million and 64% of the Internet Retailer 500. On Lynda.com you can take 828 separate courses to teach you about online analytics. Even YouTube offers more than 7000 videos on the subject. And the bottom line is that online retail analytics has become such a massive industry that the leader in the space, Omniture, was acquired by Adobe in 2009 for $1.8 billion.

But there is one key difference. According to the Department of Commerce, online purchasing represents less than 5% of retail purchasing in the United States for 2011. In other words, all this usage, content, and enterprise software licensing today represents only one twentieth of the potential for analytics to provide understanding of retail purchasing behavior. As retailers become savvy to the benefits they can derive, we can expect Applied Big Data to sweep through and ultimately transform the brick-and-mortar retail industry.

About the Author

Alexei Agratchev is Co-Founder and CEO of RetailNext, which provides real-time in-store monitoring and analytics. Prior to joining RetailNext, Alexei was the founder and General Manager of an internal startup within the Cisco Emerging Technologies Group focused on developing video applications for the gaming and retail markets. During his eight years at Cisco, Alexei held a number of leadership positions with direct responsibility for developing and launching new product lines. Prior to joining Cisco, Alexei was a consultant at Accenture in its Electronics and High Tech Operating Unit. Alexei holds a bachelor degree in International Relations from Claremont McKenna College and has also completed the Stanford Graduate School of Business.

.