Warning to those packing the shopping cart with the freshest tech for sale, such as virtualized container solutions and specialized microservices: The resulting mix is a whole different enchilada than the old nuts and bolts enterprises are used to. Microservices multiplying like germs across distributed, multicloud environments can dizzy development and operations teams.

The landscape is a bit like those messy data lakes; realistically, humans can’t tame it without analytics for categorization, permissions, access, monitoring and security.

SignalFx Inc. learned a trick or two for reigning in chaos in its formative years. The software as a service-based monitoring and analytics platform cut its teeth working with a startup characterized with an overwhelmingly busy IT environment.

“Much of our technical team were responsible for building the monitoring systems at Facebook back in the mid-2000s when they had their famous move fast and break things culture — which today everyone calls DevOps,” said Karthik Rau (pictured, right), founder and chief executive officer at SignalFX.

Today, smaller companies are stealing moves from the playbooks of giants, including Facebook. The endgame is usually agility — the ability to develop and deploy applications and detect and remediate problems quickly. Staying nimble across scattered environments requires new analytics-centric monitoring focused on identifying meaningful patterns, according to Rau.

Rau and Arijit Mukherji (pictured, left), chief technology officer at SignalFx, spoke with John Walls (@JohnWalls21), co-host of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, and guest host Justin Warren (@jpwarren), chief analyst at PivotNine Pty Ltd, during AWS re:Invent in Las Vegas. They discussed how they’re attacking microservices sprawl with data-science and analytics technology. (* Disclosure below.)

Not so micro after all

The microservice architecture method structures an application as a bunch of loosely connected services. Developers these days are building a lot of cloud-native applications with microservices and running them in containers. It’s important to note that microservices are not the same as serverless functions, though they are related. What lies beneath the service interface of a microservice are functions — single blocks of code. A function has one simple purpose — say, to process an image or translate data.

Many companies wading into these microservices and containers get swamped by the sheer number of elements. “The individual pieces are becoming smaller and they’re growing in number,” Mukherji said. “So the complexity of those interactions is becoming harder and harder to manage.”

Sixty-three percent of 354 enterprises surveyed by Camunda Services GmbH are currently using microservices. “Improved employee efficiency,” “improved customer/end-user experience,” and “cost savings on infrastructure and other development tools” are the top three business benefits, according to the survey.

The biggest challenge surveyed companies face or expect to face is “lack of visibility into end-to-end business processes that span multiple microservices,” the report stated.

“Microservices architectures provide teams with autonomy and flexibility but also introduce significant new challenges because a company’s core business processes nearly always span multiple microservices, making it difficult to gain visibility into the current state of an end-to-end process and to ensure that errors within a process are handled reliably and consistently,” said Jakob Freund, co-founder and CEO of Camunda. “Camunda’s survey makes it clear that while enterprises are adopting microservices for compelling reasons, the majority will be unintentionally limiting the benefits from the architecture and may even be impeding their ability to provide a better end-user experience.”

Sounds complicated, because it is.

The first step toward simplifying it is to achieve visibility across all microservices running across environments. Isolating a performance problem amid these different elements without visibility and tracing tools would be agonizing, according to Rau.

“It’s extremely difficult for a human to find it by trial and error across a distributed system that could involve thousands of components,” he said. “You have to go through every single trace. You have to figure out is it a particular version of code, a particular server?” 

Analytics puts a finger on performance problems

Weed-whacking complexity snafus top the agenda in the container-microservices camp lately. At the last KubeCon + CloudNativeCon event, the conversation centered around pooling angled solutions.

“It takes a spectrum,” Clayton Coleman, architect of containerized application infrastructure at Red Hat Inc., told theCUBE. “This is where an ecosystem really comes into play.”

SignalFx has announced some new features that leverage analytics for visibility and tracing.

It replaces the above-mentioned trial-and-error runaround with something called Outlier Analyzer. “Our Outlier Analyzer feature will automatically look through all of the outliers, identify the over-represented dimensions, and guide you to those specific problematic areas,” Rau said. This shortens the troubleshooting process by orders of magnitude, he added.

Its Microservices APM product gives customers visibility into the transaction flow in their microservices environments.

Its Smart Agent collects info and gives full visibility into the old and new stack of hybrid customers. Its analytics allow them to compare one versus the other. What’s different about SignalFx is its rich analytics in the back-end collecting metric data across environments, old or new, Rau explained.

“We’re able to provide very sophisticated analytics to identify meaningful patterns, outliers, anomalies and to look across all of your metadata to be able to identify whether those patterns are specific to a subset of machines or a particular version of code,” he said. “That’s typically very helpful to customers as they’re moving from the old to the new.”

Automating and cutting time to remediation changes day-to-day life in enterprises, according to Rau. SignalFx has a Fortune 500 customer that is now much more aggressive and agile in doing product launches, rolling them back and remediating issues. It’s made them more confident and eager to execute new initiatives, Rau explained.

“They’ve gone from having two rooms full of people to having just one on-call engineer every time they do a launch,” he concluded. 

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of AWS reInvent. (* Disclosure: SignalFx Inc. sponsored this segment of theCUBE. Neither SignalFx nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE