UPDATED 08:30 EDT / MAY 12 2016

NEWS

Syncsort adds native support for Spark and Kafka to help users modernize their analytics environments

A 2015 survey from Databricks Inc. found that Spark is quickly replacing MapReduce in Hadoop deployments, but adoption could be increasing even faster if not for the difficulty of upgrading. Syncsort Inc. is moving to ease the task for users of its DMX-h tool today by adding native integration with the engine that removes the need to manually rewrite analytics workflows as part of the transition.

Instead, a customer can now simply select Spark as their data processing engine of choice through the software’s graphical interface and have their implementation automatically adapted for the framework’s programming model. The conversion is then applied to every workflow created in DMX-h, including the connectors an organization uses to import records from external sources and the transformations that are executed on the information once it’s inside Hadoop. Syncsort credits the simplicity of the process to the fact that its tool doesn’t require users to do any coding in the first place.

DMX-h makes it possible to define how data should flow into an analytics cluster using high-level policies that are decoupled from the underlying engine. As a result, Syncsort says that those policies can be easily applied to different data crunching frameworks by the “intelligent execution engine” tasked with handling the implementation. The firm claims that technology is not only able to port a Hadoop workflow from MapReduce to Spark but also run it in standalone implementations of the latter, even if they’re in the cloud. Last year’s Databricks survey revealed that such deployments are becoming increasingly common as organizations look to reduce operating costs.

And at the same time, Spark adopters are increasingly making use of complementary open-source tools like Kafka, which is now natively supported in DMX-h too. That means customers can take advantage of the message broker’s performance to speed the flow of data between different parts of their analytics environments. According to Syncsort, the integration should come particularly useful for organizations dealing with sensory transmissions and other real-time information that needs to be processed while it’s still fresh. 

Image via Geralt

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU