UPDATED 09:00 EDT / NOVEMBER 21 2017

BIG DATA

Apache Spark rival Hazelcast Jet adds new APIs to boost developer productivity

Hazelcast Inc. is pitching an updated version of its open-source Hazelcast Jet big-data platform for stream processing.

The headline feature is a new application programming interface for its in-memory data grid, which is a data structure that resides entirely in memory and is distributed among multiple servers. The company said the feature can help make the platform more accessible and boost developer productivity.

Hazelcast Jet came to the fore with its initial release last February, when the company claimed its superiority over more familiar distributed processing engines for big data such as Apache Spark and Apache Flink. Hazelcast Jet can be used alongside the company’s in-memory data grid, which adds storage capabilities for incoming data streams. As a result, Hazelcast Jet can keep both computation and storage “in-memory,” enabling parallel execution on incoming data so applications can operate in real time, the company claims.

Hazelcast Jet also said it has a “one-record-per-time architecture” that allows it to process data as soon as it arrives. That’s in contrast to Apache Spark and Apache Flink, which both work by accumulating data into microbatches before it’s processed. What that means, the company said, is that Hazelcast Jet can process data much faster, as it reduces the latency of the applications it powers.

Now, with the updated Hazelcast Jet 0.5 release, the company is adding a new Pipeline API for general purpose programming of batch and stream processing. The new API is said to be “simple and intuitive” and makes it much easier to program the platform. It adds new tools for developers to create batch computations from building blocks such as filters, aggregators and joiners, the company said. The main idea is to increase developer productivity.

The new version also comes with the Java 8 Stream API, which supports functional-style operations on streams of elements. The main idea with Java 8 Stream API’s inclusion is to make the platform more appealing to Java developers, Hazelcast said.

“Since its first release Jet has put the ‘Fast’ in fast big data with performance up to 15 times faster than Spark and Flink,” said Hazelcast Chief Executive Officer Greg Luck. “In this release we have been working on bringing Hazelcast’s legendary programming simplicity to Jet, which we think we have now achieved with the Pipeline API.”

The company has also added new fault tolerance features by way of distributed in-memory snapshots, which are distributed across clusters and held in multiple replicas for redundancy. As such, the platform can now recover from multiple faults, including node failure, network partitions and job execution failures. Jet simply uses the latest snapshot to automatically restart any jobs that contain the failed node as a job participant.

Hazelcast provided the following graphic illustrating Hazelcast Jet’s architecture:

jet-diagram

Hazelcast said its Jet platform is used for applications including online trading, real-time fraud detection, system log events, sensor updates for “internet of things” architectures, and social media platforms.

Image: Hazelcast

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU