Conduktor Inc., a startup that focuses on simplifying the use of the popular Apache Kafka data streaming platform, is readying a Sept. 27 launch of its platform and says it already has a long list of pre-production customers.

Kafka is an open-source platform used by more than 80% of Fortune 100 companies to manage data streams ranging from machine sensors to stock prices and social media feeds. Allied Market Research, which is the research and business consulting wing of Allied Analytics LLP, estimates the market is growing nearly 27% annually and will reach $52 billion in 2027.

Streaming data opens up a whole new set of use cases but the software development constructs are quite different from batch processing. Streams need to be handled differently from files or records and multiple streams may be co-mingled in a single feed.

Kafka is a fast and highly scalable platform used for reliably capturing and delivering data streams. However, it has been criticized for being difficult to use and lacking a visual monitoring capability.

That’s the gap Conduktor aims to fill. “Companies need to use streaming data but they have a shortage of talent and can’t get the full value because Kafka is so complex,” said Chief Executive Nicolas Orban.

The platform is a desktop application that promises to simplify many common tasks involved in building and running Kafka clusters. A software-as-a-service version will be announced next week.

Conduktor says it can reduce the amount of time required to operate Kafka deployments by more than one-third. Its platform is said to replace a half-dozen point products that developers would otherwise have to use to work with Kafka.

It simplifies the tasks of managing topics and subscriptions, helps with managing schemas and improves data quality by filtering duplicate records and changing the data format of files to make them easier for workloads to process. It also offers a finer degree of control of how messages are received and processed.

“We run on top of everything Kafka — Kafka Connect, Kafka Streams and Kafka SQL — and bring a single productivity layer to help developers ensure that what they’re doing is working,” said Chief Technology Officer Stephane Derosiaux. “We want to show everything visually so there is nothing hidden behind the scenes.”

The company, which was founded two years ago, raised $20 million last year shortly after Kafka-based platform developer Confluent Inc. staged a successful initial public offering that valued it at more than $11 billion at the time.

Conduktor’s client software, which is not open source, has been downloaded by more than 100,000 developers in 40,000 companies, Orban said. The product is currently aimed at software developers but the company hopes to also attract data architects, quality assurance professionals and even business end-users over time.

There’s a free version that runs on a single cluster with one broker. Multicluster versions start at $45 per month per user, with enterprise licenses custom-quoted.

Photo: Unsplash