A new report released today by Kubernetes operations and cost management startup Cast AI Group Inc. finds that significant underuse of cloud resources in Kubernetes environments, which manage microservices, the components of modern applications.

That’s resulting in substantial inefficiencies and opportunities for cost optimization in cloud computing. The second annual CAST AI Kubernetes Cost Benchmark Report, based on the analysis of 4,000 clusters running on Amazon Web Services, Google Cloud Platform and Microsoft Azure, found that for clusters containing 50 to 1,000 CPUs, organizations only use 13% of provisioned CPUs and 20% of memory, on average.

For larger clusters containing 1,000 to 30,000 CPUs, organizations were found, on average, to use only 17% of provisioned CPUs.

The average CPU utilization rates were found to be the same on AWS and Azure, with both found to have a CPU utilization of 11%. Google users, though, were found to be slightly more efficient, with 17% CPU utilization, though the figure is still remarkably low. For memory, utilization differences came in at 18% for Google Cloud, 20% for AWS and 22% for Azure.

Spot instance pricing — the pricing model offered by cloud providers for purchasing unused computing capacity at lower prices than standard rates — across the six most popular instances for US-East and US-West regions (excluding government regions) increased 23% between 2022 and 2023.

The report found that the biggest drivers of waste include overprovision, unwarranted headroom, low spot instances usage and low usage of “custom instance size” on Google Kubernetes Engine. With overprovision, organizations were found to be allocating more computing resources than necessary to an application or system.

Unwarranted headroom — the practice of allocating more computing resources, such as CPUs, than necessary for an application or system — was found to be the result of organizations setting the number of CPUs they need for the Kubernetes installation too high.

Some of the inefficiencies stem from the reluctance of organizations to adopt low spot instance usage, often over concerns about their perceived instability and the potential for sudden termination. With organizations using Google’s GKE, the report notes that low usage of “custom instance size,” another feature that could reduce overheads, was the result of difficulty in choosing the best CPU and memory ratio unless the selection of custom instances is dynamic and automated.

“This year’s report makes it clear that companies running applications on Kubernetes are still in the early stages of their optimization journeys and they’re grappling with the complexity of manually managing cloud-native infrastructure,” Laurent Gil, co-founder and chief product officer of CAST AI, said ahead of the report’s release. “The gap between provisioned and requested CPUs widened between 2022 and 2023 from 37% to 43%, so the problem is only going to worsen as more companies adopt Kubernetes.”

Image: laboratorio linux/ Flickr