In-memory data grid provider Hazelcast Inc. is updating its Hazelcast Jet platform for stream processing with new features that should improve the accuracy of the data analysis it performs.

Hazelcast Jet is an open-source distributed processing engine for big data that was released by Hazelcast earlier this year. At the time, the company said its solution was a better alternative to more established open-source rivals such as Apache Spark and Apache Flink when used alongside its in-memory data grid, because it’s faster and provides lower latency than those solutions.

Hazelcast’s in-memory data grid provides storage features for incoming data streams. When paired with Hazelcast Jet, both computation and storage are kept in-memory, enabling parallel execution on incoming data so applications can operate in as close to real time as it’s possible to get.

A second difference is that Hazelcast Jet is built on a “one-record-per-time architecture” which means it processes data as soon as it arrives in the system, in contrast to Apache Spark and Apache Flink, which both accumulate records into micro-batches before processing them. This means Hazelcast Jet is able to work faster, thereby reducing latency in the applications it powers.

With Hazelcast Jet 0.4, the company is adding new event-time processing capabilities with tumbling, sliding and session windowing functionality. This should help users to get better value from the data they analyze, the company said.

Event-time processing refers to a way of partitioning data by taking fragments from the data stream and analyzing them individually. But one of its drawbacks is that events may arrive out of order or late, so you can never be sure if you see all events in a given time window. To overcome this, Hazelcast Jet now offers three kinds of windowing functionality to better evaluate stream processing jobs at regular time intervals.

In an attempt to show just how fast Hazelcast Jet is, the company also published the results of a new benchmark study comparing it with Spark and Flink. The study shows Hazelcast Jet beat its competitors with a 40ms average latency for stream processing computations, which remained flat even as the volume of messages increased. By comparison, both Spark and Flink saw latency rise at higher message throughputs.

“The new functionality in 0.4 brings stream processing for the first time,” Greg Luck, chief executive officer of Hazelcast, said in a statement. “As with batch, we are achieving a new performance level, giving us a real edge over alternative market solutions.”

Image: Philippe Put/Flickr