Vectorized Inc., a new data management startup, today disclosed that it has raised $15.5 million across two funding rounds from Lightspeed Venture Partners and Alphabet Inc.’s GV fund.

The startup’s entry into the crowded data management market is an open-source stream processing platform dubbed Redpanda that was released this morning as well. It aims to provide an alternative to the industry-standard Apache Kafka engine.

Kafka is used to power enterprise applications that ingest large amounts of small files on a near-real-time basis, an essential requirement in many areas. Systems that monitor the health of data center infrastructure, for example, need the ability to analyze server logs immediately after they’re generated to detect malfunctions quickly. Recommendation engines must deliver buying suggestions in a fraction of a second based on an online shopper’s most recent actions.

Kafka is well-suited for such tasks, but according to Vectorized, it has a major shortcoming: complexity. The startup positions its newly released Redpanda platform as a way for companies building data streaming workloads to simplify their infrastructure.

Redpanda uses the same application programming interface as Kafka, meaning it can replace the engine in companies’ existing workloads without any major code changes. Under the hood, however, Redpanda features a number of major differences.

Because of Kafka’s complexity, companies adopting the engine typically have to deploy it together with another open-source application called ZooKeeper that’s used to manage large software systems. Redpanda removes the need for ZooKeeper, meaning information technology teams have one less application to manage. Redpanda instead relies on a built-in configuration management mechanism  that automatically tunes users’ deployments to increase operational efficiency.

ZooKeeper isn’t the only source of complexity that Vectorized is addressing. Real-time applications often require the ability to make changes to the raw information they ingest, for example to remove duplicate items, which can necessitate specialized tools. Redpanda has an embedded data processing engine that performs such changes without requiring a company to set up additional components in its data environment.

Vectorized plans to make money from Redpanda by offering a cloud-based version of the platform. The offering will add a number of capabilities on top of the open-source edition’s feature set, including a disaster recovery tool and automatic infrastructure scaling.

Photo: Unsplash