Speaking with Dave Vellante and John Furrier on theCUBE at the 2014 HP Vertica Big Data Conference, Wayfair LLC‘s System Architect Jordan Chernev explained that his site differentiates itself from Amazon — “the big whale” — by using data to drive all their decisions. Internally Wayfair, an ecommerce platform, is dedicated to the concept of “data democracy” and “self-service BI (business intelligence),” so everyone in the business — engineer or manager — has “tangible access to all the data and all metrics we have collected as an organization.”

Data, Chernev said, drives department-level decisions all the way up to company-wide strategic initiatives. This type of “self-service BI helps drive growth across the board,” he went on, noting that Wayfair has been growing “50 percent year over year since 2002.”

Curious about how business users interact with the data, Vellante asked, “Is it through visualization tools? Through spreadsheets? Is it through a system that you developed?”

In their current system, the way Wayfair business users interact with data “is very Excel spreadsheet-driven. We use Microsoft analysis services, Webstack and we also have data visualization tool in the case of Tableau.”

Chernev says Excel spreadsheet manipulation interaction is used for up to 95 percent of Wayfair’s use cases. He added that they have specialized dashboards for certain business metrics that are real-time or near real time. The design of those dashboards is up to specialized business intelligence teams. The platform Wayfair uses, said Chernev, has been “in development and constant evolution ever since 2010.”

Read more after the video:

Switching from Netezza to Vertica

2010, Chernev explained, was the year Wayfair started looking for more specialized technologies for analytics. Though they started with IBM Netezza, Wayfair now works with Vertica and Hadoop infrastructure. Looking to “elevate [their] game from an infrastructure standpoint,” the decision to move from Netezza to Vertica was driven by the need to “capture unstructured data in very large volumes and very large quantities at a very high speed.”

The goal was to answer a slough of questions: “How are people interacting with the website? How can we do things like A/B testing? How does this impact revenue? What are the next priorities we want to focus on in terms of making the website more usable?”

Soon, Wayfair had a “tremendous amount of growth in the data that [they needed] to look at,” said Chernev. They were quickly outgrowing their existing infrastructure and needed to make their analytical stack “better and faster for both people who are using it and people who are working on it.”

Focused on solving speed problems, Wayfair wanted to turn data into near real-time analytics that could offer actionable business insights. They went with Vertica because it allowed them to “compress the time to deliver.”

Mobile challenges and data diversity

Furrier asked Chernev about upcoming trends in retail — particularly mobile — when it comes to user experience and predictive analytics. Using their current mobile apps as an example, Chernev said that Wayfair captures and integrates data from their mobile apps with their overall data set. Wayfair has their own custom-built data integration tool that “extracts data from relational databases and puts it automatically, on the fly, into our vertical platform.”

Chernev thinks of data diversity as an interesting challenge: “How do we take a traditional dataset and a new dataset — like the clickstream dataset, the unstructured one –” and combine them to find meaningful pieces? This mobile initiative has been around for 18 months and has achieved “pretty good penetration into the market.”

One of the biggest surprises Chernev encountered from a tech perspective was the rate of growth he’s experienced in the past few years. It’s forced Wayfair to be “a lot more proactive in terms of how we think about the next 18 months.”

Solutions won’t last 3 – 5 years any more: “Our lifespan is way more contracted,” says Chernev, “[we] always have to be on our toes and looking for ‘How do I make the next leap?'”

photo credit: gak via photopin cc