Data integration company Talend SA is quickening its march to the cloud with a free application for use in Amazon Web Services Inc. environments.

Talend Data Streams is intended to simplify and speed the ingestion of large volumes of real-time data on a self-service basis. Available in the Amazon Marketplace now, it will become part of a suite of persona-based cloud applications the company is building.

Talend aims to diversify its product line to address a growing audience of non-traditional data consumers, such as data scientists, analysts, DevOps developers, engineers and extract/transform/load specialists, said Nick Piette, the company’s chief evangelist.

“We’re trying to develop personalized views to support different types of personas, but in a way that they can collaborate and work together,” he said. Ease-of-use and cloud deployment are increasingly important parts of the equation, which Talend has been addressing by focusing on self-service and moving large parts of its data integration platform to the cloud.

Talend Data Streams rides atop Apache Beam, a  framework for building analytic pipelines across distributed systems. Pipelines constructed in Beam can be deployed with popular open-source stream processing engines such as Apache Spark and Apache Flink. Talend provides the drag-and-drop interface layer for building integration rules, which are transformed into Beam code, which then compiles into an execution framework like Spark.

“Apache Beam provides an abstracted layer across all those frameworks,” Piette said. “Data Streams provides the graphical interface over that framework so data scientists can create pipelines to ingest data rapidly. Beam abstracts away the real-time versus batch equation so the pipeline can be deployed in either setting.”

Data Streams can be used to build publish-and-subscribe mechanisms using such tools as Apache Kafka, enabling business users to subscribe to topics and get real-time reports. “Building real-time applications has traditionally required real-time knowledge,” Piette said. “With Data Streams we’re providing a persona-based interface for that function,” targeting a non-technical class of big data users that have come to be known as “citizen data scientists.”

Talend’s announcement comes just a few days after competitor SnapLogic Inc. announced a similar cloud shift with an extension that nontechnical users can employ to create complex data processes.

Image: Unsplash