UPDATED 10:30 EDT / MARCH 26 2012

3 Trends a Rearden Data Scientist Would Like to See Visualized: Q&A Series

Rearden is one of the few companies utilizing big data ideals to directly benefit the consumer.  With an edge on travel patterns, Rearden has a good idea of our behavioral patterns when it comes to planning and experiencing a trip.  With emerging tech around big data trends, Rearden is able to help businesses and end users improve that experience with fact-based analysis on actual transactions.  And with a penchant for data visualization, Rearden’s combining many of the capabilities big data has to offer in order to better present a complete picture.

In today’s Snapshot Profile, a Q&A series featuring the industry’s best experts and thought leaders, we hear from Steve Bernstein, Head of Analytics at Rearden Commerce.  He discusses Rearden’s future in big data, the top trends he’d like to see represented in data visualization, and his favorite song to sing in the shower.

What’s the most important way big data can help the consumer?

The better companies are at managing, analyzing, and acting upon findings and patterns in big data, the better they will be at understanding what consumers really want and need.  It’s a well known maxim among talented market researchers that memory is unreliable, and people are not consistent at predicting what they will want or how they will behave under a hypothetical set of circumstances.  We are what we do, not necessarily what we claim we’ll do.  Big data lets us aggregate and find the patterns in those billions of digital trails consumers leave behind, which we in turn use to present them a set of tailored choices, meeting their needs while saving them time and money.

There’s a lot of nuance in that.  Big data technologies are only enablers.  Give me analysts who are empathetic, creative, intellectually curious, and orderly, clear thinkers.  Let me point them in the right direction and give them the unbounded big data tools to satisfy their native curiosity quickly.  Do this in a company that understands how to earn the trust of its consumers to permit us to use their digital trails, and we can deliver on that promise.

Where does Rearden fit into the future of Big Data?

Our path intersects Big Data on two fronts: structured data (numerical and categorical data), and unstructured data (free text).  On the structured side, given the nature of our partners and the size of our own customer base combined with their customer bases, we expect the amount of transactional and user behavioral data at our disposal, already large, to increase dramatically over the next 12 to 24 months.  In addition, we constantly receive massive streams of inbound third party data.  It’s our responsibility to add value to these raw goods through creativity and rigorous quantitative analysis and modeling.  Big Data tools are a crucial means to that end.

On the unstructured side, we are mining user generated content, particularly merchant and product reviews, to aggregate and score attributes in new ways that are not available through inbound third party sources.  For example, we receive regular feeds from multiple sources that characterize hotels, tens of thousands of hotels, in terms of the number of rooms they have, whether they have a pool, etc., etc.—hundreds of attributes per hotel.  In addition, we have millions of hotel reviews at our disposal.  When people review a hotel or a restaurant, they talk about what matters to them, not what doesn’t matter.  We apply Natural Language Processing and Semantics techniques to those tens of millions of sentences to convert it from its unknowable unstructured state to a structured state that lets us do a much better job of understanding what matters and how well each hotel (or merchant, or product) performs on the things that matter.

What 3 trends would you like to see “data visualized?”

There is so much mingling of truth, hype, and fear mongering in the intersection of sharing personal information, privacy, and the evolution of personalization.

1.       I would love to see the truth of trends in consumer personal data sharing versus declared attitudes toward privacy.  I believe that when asked directly, consumers generally overemphasize their concerns about privacy relative to how they behave, but I’d like to see if and among whom that is true, and if and how that is changing over time.

2.       The heart of Capitalism is that we exchange our property for other people’s property in transactions where the respective parties believe they are better off for it.  We own our time and we own, well, hopefully most of our data.  When we give over our time and data, we usually don’t get the full payoff immediately.  We guess what the payoff will be and act accordingly.  I want to see the trend in “exchange rate” for time and for data in a variety of circumstances.

3.       Finally, I’d like to see what services and products consumers have come to expect to get for free and which they’ve come to expect they have to pay for over time.

You can help a consumer plan a travel itinerary with data analysis. What gives you confidence in the future?

We really are living in the Golden Age of Data.  There have been overwhelming amounts of data available use for decades (think for instance, scanner data from retail points of purchase), but until four or five years ago only the largest, wealthiest companies could afford to do anything with it.  The effective cost to analyze and wring value out has come way down such that anyone at any company can exploit Big Data.

Despite the fact that we often like to think we are spontaneous, unpredictable beings, we largely remain creatures of habit.  Habits are an efficient use of energy; they are how we as a species survived the millennia.  After all, brand does matter, right?  Whether for travel, product or service purchases as a consumer or as a procurement professional at a company, I am very bullish about our ability to do an ever-better job at recognizing those habits and personalizing offerings, even accounting for how our habits evolve and change over time.

Q&A Quickie

.

Check out Steve’s artfully crafted wooden bowls here.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU