When they first got together, cloud computing and Kubernetes seemed like the perfect couple. Now, with almost three-quarters of organizations running Kubernetes workloads in production, the honeymoon phase is over.

Kubernetes is great at container orchestration; however, its complexity and lack of transparency are causing security issues, soaring cloud costs, and delays in deployment, as well as developer frustration and burnout.

“In 2022, I expect many organizations will hit a ‘cloud-native wall,’ where complexity challenges become too big to handle manually,” predicted Matt Provo, chief executive officer of StormForge, in a recent blog post.

On Feb. 23 at 2 p.m. EST, StormForge and theCUBE, SiliconANGLE Media’s livestreaming studio, present the “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event, which will spotlight improving cloud-native production environments with machine learning. (* Disclosure below.)

Kubernetes suffers from a complexity crisis

Invariably, companies hit the “spend panic moment,” according to J.R. Storment, executive director of the FinOps Foundation in an interview with theCUBE

“They realize they’re initially spending much more than they expected,” he said.

More importantly, companies don’t have the people, processes or tools in place to be able to track Kubernetes spend and establish how to optimize their cloud costs, Storment added. The FinOps Foundation and Cloud Native Computing Foundation microsurvey “FinOps for Kubernetes” found that while Kubernetes related spend increased for 68% of companies in 2020, only 6% could accurately predict their monthly cloud bill before it arrived.

The issue is the complexity and lack of transparency inherent in Kubernetes, according to Provo. While Kubernetes provides developers with a lot of flexibility in terms of container management, there is no provision for visibility into the effect on applications as the operational parameters are adjusted.

“Often organizations are becoming more agile [and] shipping code more quickly, but then all of a sudden the cloud bill comes and they’ve overprovisioned by 80, 90%,” Provo told theCUBE at KubeCon + CloudNativeCon 2021.

This cloud waste isn’t only a bottom-line issue. Data centers are estimated to emit nearly 100 million metric tons of CO2 annually, meaning that alongside wasted financial resources, unnecessary data center use caused by overprovisioning has a heavy environmental impact.

Then there’s security. In the “State of Kubernetes Security” report, Red Hat Inc. found that security was the number one container strategy concern, with more than half the respondents experiencing delayed rollout because of security concerns. Misconfiguration tops the list of security lapses, a problem compounded by the well-documented Kubernetes skills gap. There are currently over 67,000 open positions for Kubernetes-trained engineers on Indeed.com, and the “Kubernetes and Cloud-Native Operations Report 2021” report listed “lack of in-house skills/limited manpower” as the biggest challenge when “when migrating to/using Kubernetes and containers.”

Adding to the problem, trained Kubernetes engineers are getting burned out and quitting. Fifty-one percent of respondents to a 2020 study claimed they wanted to find a new job.

Machine learning can reduce data center carbon-footprint

StormForge was formed under the name Carbon Relay in 2019 with the goal of optimizing energy usage in data centers. That mission soon expanded to machine learning-based cloud-native application performance testing and optimization. And in 2020, after the purchase of performance-testing-as-a-service platform StormForger, the company rebranded and released its platform for automated Kubernetes optimization.

Optimizing data center energy use is still high on the company’s priority list, however. StormForge has an ongoing cloud waste reduction challenge where it pledges to reduce its client’s cloud waste bills by a minimum of 30% or pay their cloud bill for them.

Rather than application performance management, StormForge defines itself as an intelligence platform. Instead of competing with APM, monitoring and observability solutions, such as DataDog, Sysdig, Sumo Logic, VMware Tanzu Observability, Dynatrace, New Relic, Prometheus, Splunk and AppDynamics, StormForge has worked to integrate its platform with these companies and counts them within its ecosystem. This means that customers don’t have to switch from their existing management tools to benefit from StormForge’s platform.

“We have machine learning capabilities that can predict very accurately what organizations will need from a resource standpoint to meet their goals, not just from a cost standpoint, but also from a performance standpoint,” Provo said.

StormForge closes the Kubernetes gap

The product announcement to be made during the Feb. 23 “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event builds on StormForge’s expertise in machine learning to enable automated Kubernetes optimization. The promise is to eliminate the barriers discussed in the beginning of this article and smooth out the path for enterprise use of Kubernetes in production

Three key gaps are blocking Kubernetes and cloud-native technology from achieving its full potential, according to Provo. They are complexity, data overwhelm, and a lack of skills. All three combine to form a perfect storm that leads to inefficiencies in Kubernetes management. StormForge promises that its new platform will address all three of these problems, using its machine learning smarts to automatically optimize Kubernetes in pre-production and production, eliminate manual trial and error application tuning, and make cloud spend transparent and easy to track.

“2022 will be the year we move closer to closing the gap on complexity, data and skills,” Provo predicts. “But it’s going to take some work, and it requires sophisticated machine learning and automation.”

Livestream of StormForge’s ‘Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning’ event

StormForge’s “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event will feature interviews to be broadcast on theCUBE. Add this event to your calendar to watch the event live. Plus, you can watch theCUBE interviews here on demand after the live event.

How to watch theCUBE interviews

We offer you various ways to watch StormForge’s “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event, including theCUBE’s dedicated website and YouTube channel. You can also get all the coverage from this year’s events on SiliconANGLE.

TheCUBE Insights podcast

SiliconANGLE also has podcasts available of archived interview sessions, available on iTunes, Stitcher and Spotify, which you can enjoy while on the go.

Guests

During StormForge’s “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event, theCUBE will talk with Matt Provo, founder and chief executive officer of StormForge, and Charley Dublin, vice president of product management at Acquia Inc.

(* Disclosure: TheCUBE is a paid media partner for the “Solving the Kubernetes Complexity Gap by Optimizing With Machine Learning” event. Neither StormForge, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Event Update!

Check out our full event coverage, as well as post-event coverage, and watch the complete event video below:

Image: StormForge