Alluxio Inc., the developer of an open-source distributed filesystem for use in data-intensive applications, today announced version 2.9 of its Data Orchestration Platform.

The new release features a new scale-out, multitenant architecture, cross-environment synchronization, improved manageability and better support for the Kubernetes orchestrator for software containers. It also boosts security and performance improvements through the use of application programming interfaces for Amazon Web Services Inc. S3 and Posix storage.

Alluxio manages data across an assortment of on-premises and cloud infrastructure, using caching to minimize latency and moving data close to the point of processing. The company targets large-scale analytics workloads using frameworks such as Apache Spark, Presto and Hive.

It has become “the de-facto choice for large internet companies to accelerate the development of their data analytics and AI applications,” Peng Chen, a big data engineer manager at Tencent Holdings Inc., said in a prepared statement.

“Customers generally have no plans of moving away from on-premises environments entirely,” said Adit Madan, Alluxio’s director of product. “They have a highly heterogeneous platform and use Alluxio for portability.”

Metadata synchronization

The scale-out architecture uses metadata synchronization across multiple environments (pictured). This enables multiple Alluxio clusters to be deployed based on workload capacity. Clusters are aware of each other and can automatically synchronize metadata among them.

This feature is particularly useful for satellite architectures in which clusters are segregated across team-level tenants for isolation. It allows the platform to scale out and add new clusters without creating a central resource bottleneck. Tenant-level isolation is enabled through metadata synchronization.

Improved Kubernetes support makes the data stack more portable. It’s enabled through an operator that uses Kubernetes custom resource definitions to configure deployment, connections to underlying storage configuration updates and uninstallation.

Authentication and access policies have been centralized across compute engines and storage via the S3 API, which is “a core element of hybrid and multicloud where data is spread across environments,” Madan said. “This is a way to access data from any source with authentication done in a consistent way.”

Alluxio connects directly to Ping Identity Corp.’s PingFederate identity and access management platform and can also work with open-source governance platforms like Apache Ranger and “anything that conforms to OpenID,” which is a decentralized authentication protocol, Madan said.

Free downloads of the Alluxio 2.9 open-source community edition and a trial version of Alluxio enterprise edition are available here.

Image: Alluxio