UPDATED 09:00 EDT / JUNE 13 2017

BIG DATA

Hazelcast Jet beefs up its real-time data stream processing capabilities

In-memory data grid provider Hazelcast Inc. is updating its Hazelcast Jet platform for stream processing with new features that should improve the accuracy of the data analysis it performs.

Hazelcast Jet is an open-source distributed processing engine for big data that was released by Hazelcast earlier this year. At the time, the company said its solution was a better alternative to more established open-source rivals such as Apache Spark and Apache Flink when used alongside its in-memory data grid, because it’s faster and provides lower latency than those solutions.

Hazelcast’s in-memory data grid provides storage features for incoming data streams. When paired with Hazelcast Jet, both computation and storage are kept in-memory, enabling parallel execution on incoming data so applications can operate in as close to real time as it’s possible to get.

A second difference is that Hazelcast Jet is built on a “one-record-per-time architecture” which means it processes data as soon as it arrives in the system, in contrast to Apache Spark and Apache Flink, which both accumulate records into micro-batches before processing them. This means Hazelcast Jet is able to work faster, thereby reducing latency in the applications it powers.

With Hazelcast Jet 0.4, the company is adding new event-time processing capabilities with tumbling, sliding and session windowing functionality. This should help users to get better value from the data they analyze, the company said.

Event-time processing refers to a way of partitioning data by taking fragments from the data stream and analyzing them individually. But one of its drawbacks is that events may arrive out of order or late, so you can never be sure if you see all events in a given time window. To overcome this, Hazelcast Jet now offers three kinds of windowing functionality to better evaluate stream processing jobs at regular time intervals.

In an attempt to show just how fast Hazelcast Jet is, the company also published the results of a new benchmark study comparing it with Spark and Flink. The study shows Hazelcast Jet beat its competitors with a 40ms average latency for stream processing computations, which remained flat even as the volume of messages increased. By comparison, both Spark and Flink saw latency rise at higher message throughputs.

hazelcast-jet-latency-tests

“The new functionality in 0.4 brings stream processing for the first time,” Greg Luck, chief executive officer of Hazelcast, said in a statement. “As with batch, we are achieving a new performance level, giving us a real edge over alternative market solutions.”

Image: Philippe Put/Flickr

A message from John Furrier, co-founder of SiliconANGLE:

Support our open free content by sharing and engaging with our content and community.

Join theCUBE Alumni Trust Network

Where Technology Leaders Connect, Share Intelligence & Create Opportunities

11.4k+  
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+ 
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.

SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.