Amazon Web Services Inc. today announced that Amazon MSK Serverless, a new serverless option for Amazon Managed Streaming for Apache Kafka, is now generally available.

Amazon MSK is designed to scale resources instantly to meet an application’s demand, vastly simplifying real-time data ingestion and streaming. The service automatically provisions and manages the resources necessary to provide on-demand streaming capacity and storage for applications.

Kafka is an open-source system for transporting real-time data that enjoys widespread adoption in enterprises. Originally developed by LinkedIn to process user activity metrics from its website, the platform is used for a wide range of applications ranging from infrastructure performance monitoring to cybersecurity.

In a blog post, Marcia Villalba, senior developer advocate for AWS Serverless, described Amazon MSK Serverless as “the perfect solution to get started with a new Apache Kafka workload where you don’t know how much capacity you will need or if your applications produce unpredictable or highly variable throughput and you don’t want to pay for idle capacity.” Villalba added the at the service is “great if you want to avoid provisioning, scaling and managing resource utilization of your clusters.”

Amazon MSK Serverless is designed for organizations that are increasingly adopting Apache Kafka to capture and analyze real-time data streams from “internet of things” devices, website clickstreams, database logs and other sources where dynamic data is continuously generated.

With Amazon MSK Serverless, Amazon claims that customers no longer have to worry about capacity planning. Amazon MSK Serverless provisions, automatically manages and scales Apache Kafka and Kafka Connect clusters to provide on-demand streaming capacity and storage for even the most variable or unpredictable streaming workloads.

Features of the service include the ability to stream data easily with Amazon MSK Serverless managing cluster capacity and Apache Kafka partitions. Customers can create an Amazon MSK Serverless cluster and stream data in just a few clicks.

The service instantly scales up and down to meet demand based on application requirements, so customers don’t need to monitor cluster capacity or reassign Apache Kafka partitions. Like all good AWS services, customers only pay for the precise capacity required for their workload with throughput-based pricing.

Talking to John Furrier, host of theCUBE, SiliconANGLE Media’s video studio in December, Rahul Pathak, vice president for analytics at AWS. said that “data’s just becoming more and more critical and top of mind for customers.”

As customers look to data, analytics and machine learning to find new opportunities, new customer experiences must be efficient, agile and flexible in an unpredictable world, according to Pathak. That’s why so many companies want to work with platformlike services that offer a huge ecosystem of data partners out there.

Image: AWS