EMC Greenplum announced at Strata 2013 a new ditribution of Apache Hadoop: Pivotal HD. Greenplum’s announcement has definitely ruffled some feathers in the Hadoop space (see our interview with Charles Zedlewski of Cloudera). However when Bill Schmarzo stopped by theCube to talk with show hosts John Furrier and Dave Vellante, the conversation stayed more on the high-level topic of optimizing Big Data.

In the data-driven world of today, data warehouses exist because they allow organizations to make better decisions, to ask questions they couldn’t ask before and to gain insights into the business said Schmarzo. To date, Business Intelligence (BI) vendors have largely stayed away from Big Data. BI tools have largely been solely optimized around SQL. However now they are going to be forced to have to explore the Hadoop platform.

There was a great question asked by Vellante in the middle of the interview: Describe how a business person traditionally would interact with that corpus of data, the data warehouse and the BI professionals, and describe your vision for how that would change with Hadoop and Big Data?

“In today’s world the BI community is dominated by reports. … the business users have been dying for the BI tools to leverage both real time data and the predictive analytics to undercover insights in the data. To provide recommendations for what they should do,” Schmarzo says.

The businesses that are going to be the most successful will use business monitoring to identify insights to improve predictive analytics and optimization. There are trends Schmarzo is seeing that support this up-level move in BI. Data is starting to be treated as an asset, and analytics is actually intellectual property. Because of both, businesses will be able to trust data to make more decisions, more quickly on a higher level of confidence.

Analytics as IP, while very forward-thinking in the Big Data realm, begs the question: is that sustainable? Or is this just a race to the mean?

“I do think it’s a race, but I don’t think it’s a race to the mean. I think it’s a race for organizations to continuously look for new ways to look at their business,” said Schmarzo.  A line that followed, that I absolutely love, is: humans are revenue optimizations machines. Big Data is soon to follow in those footsteps. Companies will constantly look for ways to identify those variables that might be better predictors of performance — Big Data will become more agile and deep analysis more like second nature.

The smart play is platforms, and smarter companies will realize that they are manufacturing these platforms organically. Platforms will move up the ‘value chain,’ according to Schmarzo, and I tend to agree. He gave a great example: Ford’s new cars havae over 1200+ sensors. Driving behaviors, car performance, customized radio, help services (OnStar), mapping devices, smart phone integration, social cues — the Big Data grab is endless in one product alone — the car.

By creating an intelligent product, the car in my Ford example, companies will in essence create the platform for other people to provide value, and valuable services and products on top of it. Big Data will become a woven fabric into the platforms of intelligent products and bi-products that support it. Most companies aren’t there yet — they don’t understand what data can do for them. But the writing is definitely on the wall.