MapR Technologies, Inc. today is adding support for persistent, stateful Docker containers along with security and performance improvements that make it easier for organizations to run Docker and analytics operations across a mix of Hadoop and non-Hadoop file systems.

The company’s Converged Data Platform combines Hadoop, Apache Spark, web-scale storage, NoSQL and streaming capabilities into one unified cluster. Its new MapR Streams product offers an alternative to Apache Kafka for publish-and-subscribe messaging with better support for global sources and prevention against data loss.

Security and performance enhancements in this release include platform-wide access control expressions (ACEs), selective auditing, and optimized support for solid-state disk (SSD) storage via highly parallelized input/output (I/O) access to NoSQL data for real-time operations.  The company also announced general availability of the MapR-DB/Open JSON Application Interface (OJAI) for JSON document database workloads.

Movable Docks

Support for stateful containers is aimed at what MapR said is the growing number of enterprises that want to run production workloads on Docker but can’t because Docker data volumes are tied to specific servers. That means that if a container fails or is moved to another server the connection with the data volume is lost.

The new MapR Posix client presents a distributed, secure, reliable, read-write file system to Docker containers to let organizations deploy data-oriented applications without worrying about lost connections to data volumes.

MapR’s platform can be used as a common data services layer to provide distributed storage for database and messaging/streaming capabilities to multiple applications. Hadoop’s HDFS file system wasn’t designed for transactional workloads and so tolerates data loss and corruption events at a level that would be unacceptable in production. A 2015 North Carolina State University study of 138 data corruption incidents in bug repositories for HDFS, Yarn and MapReduce found that only 25% of data corruption issues were reported correctly and 42% failed to generate an error message.

MapR says its MapR-FS file system provides enterprise-grade availability, recovery and protection features and can simultaneously support Docker containers, multiple Hadoop clusters and non-Hadoop data managers like MariaDB and Elasticsearch. Distributed storage support enables containers to work across multiple file systems. “When we created our data layer we eliminated the HDFS and NFS requirement and exposed not only HDFS as an API but also an NFS Posix API that updates and writes to that data,” said Dale Kim, senior director of product marketing. “Docker support doesn’t have to involve Hadoop at all.”

MapR is also enhancing administrative flexibility in this release by bundling support for Apache Myriad, a resource manager that bridges both Hadoop Yarn and Apache Mesos. The Yarn cluster manager is specific to Hadoop, so workloads running on a Yarn cluster can’t easily be distributed to non-Hadoop clusters. Myriad makes it possible for resources to be reallocated across clusters for efficiency.

Security has been improved with ACE security controls, a simpler version of access control lists, which “get unwieldy because they tend to grow,” said Jack Norris, MapR’s senior vice president of data & applications. ACEs enable security administrators to grant permissions to users and groups across data files and directories using Boolean expressions.

ACEs can now be applied across the platform and at the whole-volume level. That latter capability enables permissions to be set on an entire volume, a key requirement of multi-tenant architectures. That means there is no possibility of accesses being mistakenly granted outside of a protected volume. For example, if volume-level access is limited to the finance group, no one within the finance group can grant access to an outsider.

Auditing features have been improved with the option of output in the JSON format and the ability for auditors to selectively choose which activities to audit so that the system isn’t bogged down collecting unneeded data. Auditing now adds less than five percent of overhead to the cluster, MapR said.

Products are available immediately for download and will be available this month in major public clouds, including Amazon Web Services (AWS) Marketplace, Microsoft Azure Marketplace, and CenturyLink Inc.’s Marketplace.