A true gold mine of Big Data?  Consumer intent.  It’s why social media has become so prevalent in the marketing world — low-cost market research is great fodder for predicting what a user really wants, when and how they’ll buy.  And newer social media platforms like Pinterest have made it even easier to make those predictions, enabling consumers to outline their short- and long-term purchasing goals and desires on public forums.  Apply Big Data methodology to the mix, and you’ve got a pretty solid formula for determining consumer happiness, despite its abstract and multi-channeled origins on the web.

One thing we’ve learned in recent years is that consumers can’t be categorized as absolutely as historic demographic pools have allotted — we all wear many hats, face diverse circumstances and act on different motives.  Businesses and marketers are realizing our multi-faceted intentions, finding new ways to provide Consumer Services without the limitations of pigeon-holing.  And one company that’s determined to measure consumer happiness through the use of Big Data technology is BloomReach.

In today’s Profile Series we hear from Ashutosh Garg, the CTO and Co-Founder of BloomReach, who outlines exactly how his company goes about measuring consumer happiness.  Ashutosh also speaks on educating web publishers on the importance of Big Data when it comes to providing consumers with relevant, long-tail content, and how software helps BloomReach scale its Big Data analytics solutions.  Given the data center’s shift to software-led solutions, Ashutosh’s insight on this topic is especially timely.

 How do you see, and build around, the relationship between consumer intent and content relevance?

It’s important to measure, track and analyze against a number of data points, as opposed to looking at them in isolation. At BloomReach, we look at the content source of where the customer is coming from – Were they looking for a digital camera on a review website?

Then we identify and analyze what channel the customer came from. This could be from  social media, a mobile device, a search engine or email. Users from each channel exhibits very different behavior and expresses much different intent, though it’s not always good to assume that all traffic from social media will act a certain way. However, tracking what they do when they get there adds to the collective insight.

If the user comes from a device, another point to identify is which device. Consider questions like “what is their bounce or conversion rate once they arrive?”

Aside from looking at the points of origin for users, have to consider what their past behavior was when they landed on a page. How many pages did they view? Where did they bounce – a category page? Do they usually purchase sale items?

These are just a sample of the types of elements we consider when determining and predicting intent. When you think about the millions of different permutations of each of these data points, and then multiply that across all searchers, you realize that this is a huge big data problem. You must build an engine that can feed itself, learn against past behavior, analyze what words are associated with which products and what are the likely synonyms for the product descriptions.

Here’s a simple example. Say a person is looking for that digital camera, they clicked on a product listing add from a laptop with only a Cannon digital camera with a promotional discount, but bounced off a page with $1,000 cameras in the past, but previously, they bought a similar camera that was on sale. All of these elements and behaviors exhibit intent that can be captured, processed and tracked to present something like a “You might like this” widget.

Many times, companies only look at pieces of the intent, one or two elements. We consider all of them, piecing the data together to make the most relevant experience and suggestions possible.

Can Big Data solutions help BloomReach measure something as abstract as consumer happiness?

Absolutely. Addressing and measuring customer “happiness” is like diagnosing a rare disease – while individual variables (symptoms) may indicate specific conclusions, observing and measuring them in isolation or without perspective can lead to the wrong diagnosis. The one thing we do know is that happy customers buy things from you. But, how do you gauge “happiness?” You can’t see the smiles or leaps of joy through your analytics dashboard; plus, does a happy customer definitely mean you’ve created a quality page?

My advice is to use the data you do have to measure quality by a set of factors that indicate customer happiness. And, in most cases, that task requires a big data strategy. Big data’s role in determining happiness is anchored in understand relevance, reducing noise and identifying where to focus.

Some of the key data signals that influence relevance include bounce rate, time spent on a site, number of page views and conversion; but – like that rare disease – looking at each one in isolation may lead to the wrong conclusion. For example, decreasing a metric like bounce rate may appear to be a good thing, but that doesn’t always mean you have quality pages with happy customers finding what they want.

Increasing the number of page views per visitor or time on a site may indicate that people are eagerly exploring your content, but what if users are spending 20 minutes bouncing from category page to product page and back to category page? Chances are you have a lost, frustrated and unhappy customer. In addition, consumers are increasingly using their smartphone to shop, but many mobile sites have no concept of a page view because of the continuously scrolling functionality. So if 15 percent of your traffic is coming from mobile, optimizing a site based on page views will ignore the fastest growing segment of you customers.

Like bounce rate, there is a similar threshold for conversion rate versus the number of page views. In the example below, the conversion rate rapidly increases, but stalls abruptly and actually begins to dip after an average of 10 page views, so it’s important to identify the inflection point and optimize for that metric.

There’s been a lot of talk around the hype of big data and when the investment is necessary, and I will say that it isn’t something for everyone. For example, if you are bidding on five keywords, have five pages and are trying to get five users, that’s 125 combinations – which in theory can be analyzed and acted upon manually. However, in order to compete effectively in today’s ecommerce climate, one must acknowledge that the sheer volume of information it takes to effectively compete is almost assuredly a big data problem. Big data technologies help you target appropriately, create relevant content, thoroughly analyze consumer intent, address behaviors like page views and bounce rate – which all factor into happy customers.

You can bet that sites like Amazon and Walmart.com are slicing the data in every which way to ascertain actionable intelligence, and I think we can all agree that the amount of available data will only increase exponentially over time.

Big Data has become an accepted concept in the retail industry, but how do you get publishers to fully recognize, and apply, the full potential of Big Data solutions for actionable insight that could make them more competitive?

People recognize that they need more data, but the problem is that they don’t have resources, or that the conclusions about their data aren’t actionable. Many cite the reason that it is because it is too hard to do. There right, it is extremely hard to make use of the data, and large ecommerce giants like Amazon with billions of dollars in their technology war chest are up against you. By exposing the APIs, BloomReach makes it easy to use the data and compete with the likes of Amazon and Walmart.com.

 When it comes to scalability, what can BloomReach provide publishers in terms of the right architecture and software solutions to fully leverage analytics?

When you look at a lot of analytics companies, they come in sort through unbelievable amounts of data and come back with you in two weeks with recommendations. That is simply too long. The world has probably changed and you have missed your opportunity by this point. It has to be in near real-time. At BloomReach, we process about 1,000 and five terabytes of data a night. Then, our big data application takes action, and allows marketers to make final decisions very quickly. Our Web Relevance Engine combines machine-learning with natural language processing, making it a continuously updated and smarter machine each day. In addition, we realize the value that humans bring to the table. By merging our algorithms with our staff and our customers’ expertise, we think we provide a pretty fast and reliable system to compete.