As the BigDataNYC 2016 event kicked off at the Mercantile Annex in New York, NY, Dave Vellante (@dvellante), Jeff Frick (@JeffFrick), Peter Burris (@plburris) and George Gilbert (@ggilbert41), cohosts of theCUBE, from the SiliconANGLE Media team, sat down to discuss what could be expected from this year’s event coverage.

In particular, the analysts discussed the recent trends in Big Data toward Artificial Intelligence (AI) and machine learning, becoming increasingly attractive to many companies in the enterprise.

Hadoop complexity is driving innovation

AI and machine learning with Hadoop has always been a focus area of Big Data, but recently it appears that more companies are moving away from Hadoop in order to bring more productivity and value to the datasets, according to theCUBE analyst team. The main reason for this was because developers wanted more flexibility, which in turn brought unexpected complexity and then a new wave of innovation.

“There is this new wave of innovation, a new wave of opportunity,” explained Frick. “A ton of people on the panel last night, some of the leaders in the Hadoop space … had left to start new companies around new opportunities in real-time and advanced analytics.”

George Gilbert further elaborated on this point, saying that developers “took out the pieces so you could do a mix-and-match assembly and get a pipeline of analytics that was just right for you. The unintended side effect was a lot of complexity.”

Creating new tools will underscore the value of Big Data

Much of the new innovation in Big Data, along with the challenge it has presented, is aimed at creating tools to better apply machine learning and deep analytics in order to extract value from huge datasets, theCUBE team explained. Once this is done and new tools and standards are created, companies can then focus on using them instead of creating them, and the enormous value of Big Data will come to fruition.

“What companies are really trying to do with Big Data technologies is liberate the value of data,” said Burris. “This stuff is going to be some of the most important stuff that has ever come out of the computing industry. The whole concept of Big Data is incredibly important, and it’s going to have long-lasting effects on not just this industry, but on all industries.”

Watch the complete video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of BigDataNYC 2016.

Photo by SiliconANGLE