There were two big takeaways I had from Nate Silver’s time on #theCUBE during this week’s Tableau Customer Conference: 1) the editor of ESPN‘s FiveThirtyEight is more of a skeptic, taking a wait-and-see approach to Big Data, and 2) culture is more important when analyzing data than most realize. I was fascinated to hear Silver talk about culture being key for good data analysis, and was equally intrigued to learn that he isn’t as bullish on Big Data trends as most in his field tend to be.

In Silver’s keynote, he spoke on how the volume of data does not trump the modeling of data. It is not your typical “more the merrier” situation — good data analysis requires both. The interesting part to that equation is that there is a third factor: volume + modeling + culture = good data analysis. “You need volume of data, you need high quality data, you need a culture that actually has the right incentives aligned, where you really do want to find a way to build a better product to make more money,” said Silver. And that is where my interest was thoroughly piqued.

Silver alludes to the concept that large organizations have people within the company that are thinking much more short-term than the C level executives. Furthermore, different departments in the company might be thinking about their P&L only, as that is what measures their level of success.

“A lot of success in business, and certainly when it comes to the use of analytics, is just stripping away the things that get in the way from understanding and distract you. Its not some wave a magic wand and have some formula where you uncover all the secrets of the world. Its more like, if you can strip away the noise than you’re going to have a much clearer understanding of what’s really there.”

Maintaining healthy skepticism

Being known as a Big Data guy (whatever that means), you might be shocked to learn that Silver takes issue with some of the core notions of Big Data too. Specifically, the importance of causation vs. correlation. Another recent guest of #theCUBE with a strong opinion on causality was Kenneth Neil Cukier from the Economist. In his book Big Data – A Revolution That Will Transform How We Will Work & Think, he states that causality really doesn’t matter anymore. In summary, if you know that your customers are going to buy more products by analyzing a data set or correlation, then the “why” doesn’t matter — you should try to exploit that.

Watch the interview with Silver in its entirety below.  More on Silver’s segment after the clip:

 Causality matters

Silver disagrees firmly with the notion that causality doesn’t matter. He recommends hypothesis testing.  Coming in with some questions and looking for that causality are very important to data analysis and predicting. This is where his bearishness on Big Data and the current trends being predicted shows its teeth.

“If it’s true that merely finding a lot of correlations and very high-volume data sets will improve productivity, then how come we’ve had such slow economic growth over the last ten years? Where is the tangible increase in patent growth, or different measures of progress?”

Silver is suspicious of people saying that one technology is a game-changer relative to our entire economic history. Big Data is no free ticket to success. Looking at Big Data and the Internet of Things together, I now have calmer approach to my technology prophecies. The understanding and humanizing of data is going to change our world for the better…but I’m not sure we’re even close to realizing what that means yet.