Big-data company data Artisans GmbH is enabling online transaction processing on streaming data through its commercialized version of the Apache Flink platform.

Apache Flink is an open source stream processing framework that was created to process rapid flows of real-time data, such as that produced by credit card activity monitoring, machine learning and business intelligence.

Stream processing, in a nutshell, enables users to query continuous streams of data and make informed decisions based on that information, in real-time.

Data Artisans is the company founded by Apache Flink’s creators to commercialize the software. It launched its data Artisans Platform, or dA Platform, in preview back in 2016, before finally hitting general availability earlier this year. The platform, which can process data in true real time without any latency, is designed to power “high-performance streaming applications” that need this capability, and has already seen adoption at companies such as ride-sharing firm Uber Technologies Inc.

Now, data Artisans is adding a new capability to the dA Platform by enabling more complex, “ACID” (Atomicity, Consistency, Isolation and Durability) transactions on stream processing data. ACID refers to a set of properties for multistep database transactions that are intended to guarantee validity in the event of errors, power failures and the like. An example might be a transfer of funds from one bank account to another that involves multiple changes such as debiting one account and crediting another.

ACID has up until now only been available on older, relational databases such as the Oracle Database or Microsoft SQL Server. Streaming frameworks such as Flink were previously only about capturing data, and didn’t allow for the creation of key attributes necessary for business transactions that ACID defines.

The ACID capabilities are on the dA Platform thanks to a new component called the data Artisans Streaming Ledger, which adds “multi-row, multi-state, cross-stream transactions” to data stream processing. The end result is that dA Platform users can run applications without an underlying database that are better able to scale and handle tasks such as pricing, billing, inventory management, supply-demand-matching and logistics, the company said.

“Guaranteeing serializable ACID transactions is the crown discipline of data management,” Stephan Ewen, co-founder and chief technology officer at data Artisans, said in a statement. “It is a very hard problem – something that even some large established databases fail to provide.”

Analyst Holger Mueller of Constellation Research Inc. acknowledged the magnitude of the problem data Artisans is trying to solve, saying that if the new Streaming Ledger is working and reliable, then it is a breakthrough of “historic proportions” because the ACID capabilities of databases are critical for business applications.

“So far this has been the domain of relational databases, which have been powering business applications since the 1970s,” Mueller said. “Bringing ACID qualities to streaming databases enables a new set of next-generation applications that were previously impossible due to the inferior performance of relational databases. The power of streaming databases is much closer to the speed demanded by businesses today, so these are truly exciting times.”

The new Streaming Ledger feature is bundled with the “River Edition” of the dA Platform, available now. The company also offers a less-featured version called the “Stream Edition” that includes its Application Manager, which provides tools that streamline and simplify the deployment of real-time data streaming apps in production.

Image: TheDigitalArtist/Pixabay