Wikibon analyst Jeff Kelly and theCUBE hosts John Furrier explored the shifts in the Big Data industry that Amazon is now creating with it’s existing and new cloud services and tools, live at the AWS re:Invent conference.

“AWS pretty much has the entire Big Data workload taken care of,” Kelly said, explaining that Redshift successfully takes on the traditional data warehousing world, EMR is taking on the open source Hadoop world, while DynamoDB is growing fast in the database segment.

“If they close the loop with Kinesis, who can take on Amazon? Who can provide social on top of the infrastructure,” Furrier asked.

The next challenge for AWS in the enterprise

“Redshift is the fastest growing AWS product in terms of revenue,” Kelly stated. The technology is analytic data base, known for performance. “Amazon can scale now to petabyte size. They’ve got the full stack of BD products.” The challenges they will have to face are integrating, orchestrating them, getting companies to buy into moving data to the cloud, especially sensitive data.

Commenting on Cloudera’s current position, Furrier pointed out it was “a movement that’s about who can deliver the platform solution. Amazon has the leverage of the scale,” which changes the game for Cloudera, turning it from a big player into to a small player. He asked what Cloudera needed to do to close the loop as Amazon had.

New challenges for market rivals

Cloudera is “a one-product company,” Kelly explained. “They need to start embracing the cloud more,” as currently the majority of deployments are inside the firewall.

With Hadoop, infrastructure is a challenge. Amazon takes care of that problem for its customers, Furries states. Hortonworks is pure open source, “their business is growing every day, I think they are going to make it.” Cloudera could be a cloud player. If the cloud shifts, they could compete with Amazon. Cloudera could be the catalyst in competing with Amazon. Another possible competitor is VMware with shift of the game to IT, as “Amazon has not cracked the nut of IT.”

Cloudera, “if they were going to make that kind of pivot, embrace the cloud, they have to look at something like OpenStack. They need to partner with other companies in the open cloud. My advice to Cloudera would be look at partnering in the cloud” to deliver a full enterprise platform, Jeff Kelly said.

“Amazon is moving the ball one yard at the time,” Furrier said. “The big passing play, the big yardage game is Kinesis. That streaming piece connects the dots, closes the loop.” Once the loop is closed, Amazon will be able to go back to harden and scale.

Amazon must now convince customers to adopt cloud

The challenge will be the culture issues around cloud adoption, according to Kelly. Kinesis needs to prove it with use cases. “If they can show proof points on what they can do in the Industrial Internet,” they can convince customers.

The opportunity to compete with amazon will come from a wild card like Cloudera, says Furrier, or from VMware. “If the enterprise accepts the DevOps mindset, the tooling will change.”