Continuing a recent push into the real-time data processing that began with the 5.0 release of its namesake Hadoop distribution, MapR Technologies Inc. today debuted a converged messaging and data integration platform that it says simplifies the delivery of streaming data by using a single pipe that can access multiple data sources and deliver to multiple output formats.

The MapR Converged Data Platform is essentially a message queuing engine that can handle sources ranging from real-time Internet of Things streams to structured relational data. It combines file, database, stream processing and analytics to pull data from a wide variety of sources and deliver information on a publish-and-subscribe basis to people and machines. For example, data can come from a combination of sensors, newsfeeds, log files and database queries and be delivered directly to user dashboards, analytics engines, report generators and batch database engines from a single stream depending on how subscriptions are defined. A novel twist is that the data stream itself can be saved and replayed, making it an auditable system of record.

The company says its approach solves a growing problem for organizations trying to integrate legacy data sources with a wide variety of new-age tools ranging from Hadoop batch analytics to Apache Spark and Flink real-time processing. Because of format inconsistencies, data originating from disparate sources needs to be duplicated and stored in different processing engines depending upon intended use. That slows data delivery, creates redundancies and limits the ability of users to get at the data they need.

“Our focus is to support all processing engines with a data layer that supports, file, real-time and batch in one consistent way,” said MapR Chief Marketing Officer Jack Norris.

MapR says one of its beta test customers – a global advertising technology company – has been using Streams to deliver customized views of real-time advertising data to employees and external clients across the globe. Previously, reports were delivered daily in batch, but customers can now access real-time click streams, perform analysis and change their advertising programs dynamically.

The idea is to move intelligence about data into the stream so as to make the source data format a non-issue, Norris said. MapR uses JSON (JavaScript Object Notation) – which is known for its flexibility and adaptability – as a common data interchange layer. “It’s polyglot persistence. You can have all kinds of different data working together,” Norris said.

Underlying the messaging layer is the core MapR engine and the MapR-FS file system, which provide auto-failover and order consistency, ensure cross-site replication and support unlimited persistence of all messages in a stream. The company said its platform is scalable enough to handle billions of messages per second with full data recovery.

With availability set for January, Streams will be offered as an optional add-on module to the MapR Enterprise Hadoop Platform. Pricing starts at $4,000 per node.