Syncsort open-sources Spark connector for mainframes

The clash between old and new gripping the analytics world moved up yet another notch this morning with the release of a connector from Syncsort Inc. that aims to make it easier for organizations to tap records in their mainframes using Apache Spark. It’s being made available under the same open-source license as the data crunching engine.

The launch comes two months after IBM Corp., the leading producer of big iron, committed a cool billion dollars and some 3,500 engineers to accelerating the development of Spark. The two technologies represent two entirely different eras in data processing that have been in desperate need of bridging.

That’s because many of the mainframes still operating today are found in the financial sector, which is one of the leading adopters of modern analytics technologies such as Spark. In fact, one of the top use cases for the engine is finding anomalies in transactions, which often as not pass through an IBM system during processing.

The new connector will allow organizations to feed financial logs and other important data from their mainframes into Spark without the manual integration needed otherwise, saving time and effort. That information can then be integrated with other records to provide a broader view of operations.

But Spark is hardly the only tool in enterprises’ analytics arsenal, which is why Syncsort released a similar connector for Apache Sqoop, a likewise open-source technology meant to automate the migration of data from relational systems and Hadoop, last year. The pair will likely be joined by more integrations in the future as the company continues to work on simplifying the process of moving data out of aging mainframes.

Photo via IBM

A message from John Furrier, co-founder of SiliconANGLE:

Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.

Join Our Community 

Click here to join the free and open Startup Showcase event.

“TheCUBE is part of re:Invent, you know, you guys really are a part of the event and we really appreciate your coming here and I know people appreciate the content you create as well” – Andy Jassy

We really want to hear from you, and we’re looking forward to seeing you at the event and in theCUBE Club.

Click here to join the free and open Startup Showcase event.