In the era of Big Data, some may say what was old is new again. When it comes to data, the notion of collecting it has always been around since the old IBM days. Today, with the increase in available data, there is also a wide variety of options available as well. While technologies like Apache Hadoop offer upfront cost savings for housing your data, what are the other costs you should be considering when integrating your data onto a new platform?

James Markarian, CTO of SnapLogic Inc., a software provider that offers Integration Platform as a Service (IPaaS) tools for connecting data to the cloud data sources, knows the price of moving data. He discussed the evolution of databases and the current unknowns of data integration with George Gilbert (@ggilbert41) and Dave Vellante (@dvellante), cohosts of theCUBE, from the SiliconANGLE Media team, during the BigDataNYC event.

The database evolution

Markarian began the interview talking about the explosion of data and spoke about the marketplace and how it has radically grown over the years. ”When you look back 10 or 20 years, you could pick any flavor of database … as long as it was Oracle, IBM or Microsoft. Now you have relational databases; you have NoSQL databases, and then you have purpose-built databases for doing analytics so everything from Vertica [HPE] to Redshift [Amazon] and, of course, Hadoop; it’s exciting for us in technology,” he observed.

However, with all the new and exciting technology to choose from, the customer is left with the challenge of implementing it. Markarian said that almost everything has changed, from the platforms to people’s expectations about the value of their data. He also noted that the complexity has gone up considerably and it is now both challenging and exciting.

Minting IT scientists

According to Markarian, the issue with all this evolution is that things don’t change overnight. “We didn’t mint IT scientists or people with the technological skills all of a sudden,” he commented. He goes on to say that the while some technology is cheaper from a hardware and software perspective, there is still “people cost” — the cost of hiring individuals who can make it work while keeping up with the latest technology.

He does see a potential solution coming. “I don’t know if you heard about this thing; it’s called the cloud. It’s going to be big,” he feigned. He believes that many of the improvements in the app space for transnational applications are going to be moving to the cloud, making it simpler.

Helping to make the move

SnapLogic offers an IPaaS solution for customers who want to integrate applications and data in the cloud. The platform, hosted on Amazon Web Services, allows customers to deploy on-premise or in the cloud, and the data stays where the customer wants it.

The company is also incorporating data integration for analytic purposes and Hadoop integration in cloud and on-premise, Markarian said.

Image: theCUBE