Open-source unicorn Confluent Inc. is ready to go head-to-head with cloud computing giants with the release of a cloud-native and fully managed service based upon the Apache Kafka streaming platform.

The announcement today fills an important gap in Confluent’s product portfolio. Although the company has sold its streaming platform as a cloud service for some time, the offering was basically a cloud-enabled version of its on-premises platform. The new cloud-native version is more scalable and introduces variable pricing based upon consumption, which is likely to reduce costs for most cloud customers, according to Gaetan Castelein, Confluent’s vice president of product marketing.

Launched in 2014, Confluent made waves early this year when it announced a Series D funding round of $125 million, valuing the company at $2.5 billion. The event was notable because Confluent sells software built upon an open-source base, a business model that has been notoriously difficult for software companies to make money from.

The move will step up Confluent’s “coopetition” rivalry with cloud infrastructure-as-a-service providers that sell their own versions of Kafka while also supporting Confluent’s. Confluent stirred up controversy last December by announcing changes to its licensing provisions that prohibit cloud providers from delivering its software as a service.

Kafka is an event streaming platform that can reliably juggle massive volumes of real-time data for applications ranging from ingesting website traffic metrics to collecting soil measurements recorded by sensors in a far-flung field. Like many open-source projects that were rooted in on-premises development, the open-source version of Kafka didn’t smoothly make the transition to cloud-native deployment, Castelein said.

“Most weren’t designed to be consumed natively in the cloud, so they aren’t elastic, don’t scale well and can take days to provision,” he said. “To be truly cloud-native, a service should be self-service, provision instantly and scale on demand.”

All major cloud platform vendors offer their own versions of Kafka as a service, he said, but this will be the only truly cloud-native Kafka service out there. “No other instance has this ability to scale elastically,” he said.

Specifically, the company said Confluent Cloud can scale dynamically between zero and 100 megabytes per second of throughput in seconds and support gigabyte-per-second deployments with planned provisioning. Consumption-based pricing makes it possible for customer to pay only for what is actually streamed. Previously, pricing was based upon peak loads, which caused some customers to pay for capacity they weren’t using.

The company is also introducing three managed services based upon the needs most often identified by customers. They include a managed central registry to define and share standard schemas for events, a streaming SQL query engine and automated connections to the most widely used data sources, the first of which is Amazon Web Services Inc.’s S3 storage.

Pricing starts at 11 cents per gigabyte of data ingested into or transferred out of the cloud.

Image: Pixabay