UPDATED 12:49 EDT / DECEMBER 17 2019

CLOUD

Q&A: ETL and data pipelines take a leap into the as-a-service model

There are more data sources than ever before, mainly thanks to the cloud. And having all this data stored within an organization’s data center is useless if it gets siloed. Having stored data is like having a raw and uncut diamond — yes, it is still a diamond, but it looks like a rock!

Data needs to be exposed so that it can be transformed and loaded for analysis and modeling, according to Christian Romming (pictured), founder and chief executive officer of Etleap Inc., an ETL and data pipeline company. Making sense of data is like shaping the raw diamond for a setting. And this is where the whole extracting, transforming and loading process comes into place; it makes data ready for analysis. But ordinary ETL is complicated, expensive, and requires a sophisticated infrastructure. Etleap is leveraging the whole cloud computing model to deliver ETL managed as a service.

Etleap is based entirely on AWS’ infrastructure. Data is loaded into AWS’ Redshift and Snowflake warehouse raw-database. With this model, data engineers can quickly load their data into their data lake to prepare and refine it with the ultimate purpose of building data models, analytics, or visualizing data, Romming explained.

Romming spoke with Dave Vellante (@dvellante) and John Walls (@JohnWalls21), co-hosts of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, during the AWS re:Invent conference in Las Vegas. They discussed ETL as a managed service and data pipelines. (* Disclosure below.)

[Editor’s note: The following has been condensed for clarity.]

Walls: Let’s talk about Etleap a little bit.

Romming: Etleap is a managed ETL-as-a-service company. ETL is “Extract, Transform, and Load, [and is] basically about getting data from different data sources, like applications and databases, into a place where it can be analyzed, typically a data warehouse or a data lake. 

Walls: What are you seeing, big picture-wise, in terms of what people are doing with their data, how they’re trying to access and derive more value from it?

Romming: There are a few trends we see these days. Companies used to have data warehouses on-premises, and now they’re in the cloud. They are cheaper and more scalable. And then another trend is that companies have a lot more applications than they used to. Nowadays, companies have hundreds or even thousands of applications that effectively become data silos. … Analysts are seeing value in that data, and they want to have access to it. 

Vellante: The challenge with ETL has always been that it’s cumbersome and expensive, and now we have this new cloud era. How are you guys changing ETL? 

Romming: We started Etleap because we saw that ETL is not going away. In fact with all these applications and all these new needs that analysts have, it’s actually becoming a bigger problem than it used to be. And so what we wanted to do is basically take some of that pain out so that companies can get to analyzing their data faster and with less engineering effort. 

We’re specifically 100% AWS, so we’re deeply focused on Redshift, data warehouses, S3, and glued data lakes. And so what we do is we take the infrastructure piece out, so you can deploy Etleap as a hosted service, where we manage all the infrastructure for you. Or you can deploy it within your VPC, in a simplified way compared to traditional ETL technologies. 

Vellante: Do you see an opportunity to actually create solutions for the enterprise? Or is that antithetical to the AWS cloud model? What are your thoughts on that? 

Romming: Now that we’re sort of moving upmarket and into the enterprise, we’re seeing that they have requirements that go way beyond what venture-back tech needs in terms of security and governance. In ETL, specifically, that manifests itself in terms of not allowing data to flow out of the company’s virtual private cloud, for example. Data lineage, is another one. Understanding how data makes it from all those sources into the warehouse, what happens along the way. In regulated industries, in particular, that’s very important.  

Walls: What do you see the next 12, 18, 24 months, as far as where you want to focus or what you think your customers are going to need you to focus on?

Romming: Bigger and bigger companies now are realizing that there is a ton of value in their data and in all these applications. But in order to get value out of it, you have to put engineering effort into building and maintaining these data pipelines. And so our focus is on reducing those engineering requirements. So we believe that a lot of that, which has traditionally been done with specialized engineering, can be done with great software. So that’s what we’re focused on building. 

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of the AWS re:Invent event. (* Disclosure: Etleap Inc. sponsored this segment of theCUBE. Neither Etleap nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU