Data integration provider Talend Inc. is tightening security, improving self-service features and adding support for the latest release today of the Apache Spark analytics framework in a new version of its Talend Data Fabric.

Talend now integrates natively with Spark 3.0, which was released just over a year ago. Among the enhancements in that release are performance improvements enabled by adaptive query execution and dynamic partition pruning, compliance with ANSI SQL and improvements in the application programming interface to pandas, a popular open-source data science and analysis package.

The new release of Talend is also compatible with Spark implementations from Databricks Inc. and Amazon Web Services Inc.’s Elastic MapReduce 6.2.

“Spark provides new primitives to integrate at scale,” said Thomas Steinborn, vice president of product at Talend. “This means you’ll generate the most performant code to get the most out of the data you have.”

Talend is also adding a self-service portal that offers a central listing of application program interfaces and documentation. The portal makes it easier for end-users to discover resources and for developers to share APIs, documentation and data structures across a team.

“The focus on the self-service piece is about enabling everybody in the organization to participate in data governance,” Steinborn said. “What’s new in this release is providing end-users with the right user experience, making sure operations are repeatable and operationalizing the data integration pipeline.”

Talend also said it’s the first data integration provider to support private connectivity between its platform and AWS or Microsoft Corp. Azure instances via a private link, which is a more secure alternative to a virtual private network.

“A VPN opens up the entire network and exposes your entire company,” Steinborn said. “A private link just exposes one dedicated endpoint in the organization. Instead of exposing the entire company we’re exposing only one database. We want to provide more choices for our customers who may not want to send data over the public internet.”

A private link also reduces data transfer costs and increases performance because it uses a low-latency network connection and avoids hops on the public internet. It also complies with local data protection and industry regulations such as the Payment Card Industry data security standard and the Health Insurance Portability and Accountability Act.

Image: Pixabay