New Relic enhances Kubernetes observability with open-source Pixie software
Observability platform provider New Relic Inc. is beefing up its product with a new “Kubernetes experience” that it says will enable instant insights into the software container orchestration system without needing to update any code or sample data first.
The company also announced enhancements around its error tracking, network monitoring and programmability capabilities, plus two new community-focused versions of its platform. The announcements, during New Relic’s annual developer conference FutureStack 2021, are meant to help customers adopt a more data-driven approach to observability that informs how they build, operate and deploy new software.
New Relic sells application and DevOps monitoring tools that help developers detect problems with their software applications. It basically provides a monitoring service for apps that runs in real time, helping DevOps teams gain a full understanding of any issues that could affect software performance.
The New Relic One platform is popular, but the company has struggled to turn a profit. Like many tech firms, it has moved its business away from software license sales to a subscription model, and last year it announced a major overhaul of its product pricing to try to differentiate its value proposition more. That was seen as a necessary move by the company’s management, since the observability market is rapidly becoming more crowded with competing offerings.
Earlier this month things came to a head when New Relic announced that Chief Executive Officer Lew Cirne had stepped down, though it did install a ready-made replacement in industry veteran Bill Staples.
For all of its troubles, New Relic has continued to focus on delivering a better experience for customers, and its new Kubernetes experience represents the fruit of those efforts. The new capability is available in beta starting today, powered by Auto-Telemetry with Pixie, a technology the company got its hands on when it acquired a startup called Pixie Labs Inc. in December.
The key advantage of Pixie Labs’ technology is its ability to gather operational data from Kubernetes, which is open-source software used to monitor software containers that host the components of modern applications, without writing any extra code. That saves time for developers, especially in large enterprise software projects where dozens or hundreds of components may have to be instrumented.
“Instrumenting an application to provide the telemetry data you need was a highly manual process,” Zain Asgar, who co-founded Pixie and is now general vice president and general manager of Pixie at New Relic, told SiliconANGLE. “It involves lots of code changes and could take weeks, if not months, to see value. Even just maintaining this instrumentation was a large burden for teams.”
Pixie gets around this by using a Linux kernel technology called extended Berkeley Packet Filter, Asgar explained. The eBPF works by capturing deep information about what’s happening inside applications by tapping into network traffic, without changing any code within the apps. “This instant visibility includes full-body requests, service maps, and rich latency and error statistics, all of which would have taken weeks or months to collect in the past,” he said.
Pixie’s software addresses some other problems around Kubernetes observability too. Asgar said that one of the problems with existing tools is they export their telemetry data to a remote cloud, which can be burdensome for critical production networks and also expensive given the sheer volume of the data.
“Rather than exporting telemetry data to a remote cloud, Pixie stores data directly on your cluster,” Asgar said. “This is helpful from a privacy and security perspective because sensitive telemetry data remains in place, and it also enables more data to be collected without causing undue load on production environments.”
Another challenge with existing observability tools for Kubernetes is they were mainly designed for ad hoc usage rather than for programmatic access, Asgar pointed out, which means it cannot be leveraged for automated developer workflows such as continuous integration/continuous development.
“Pixie is a fully scriptable platform [that] provides an API to query data that can be integrated into automated workflows,” Asgar said. “We’ve seen users set up Slackbots on top of the Pixie API, for example. Our query language is rich and allows for flexible, custom analysis.”
Another advantage of Pixie is that although it rivals other observability tools, it can also play nicely with some of them to deliver even more visibility into Kubernetes workloads. “We think that it serves our users to play well with other systems, which is why we built Pixie to easily import and export data to and from other tools,” he said. “We’ve already built out a Grafana plugin so that Grafana users can leverage the data Pixie collects, and there will be more integrations to come.”
Those integrations may come soon because New Relic recently announced it would contribute the Pixie open-source project to the Cloud Native Computing Foundation. By housing the project under the CNCF, it’s likely that hundreds if not thousands of new developers will contribute to help improve it.
“Kubernetes remains very hard to manage, so it’s good to see that New Relic is making it easier to observe clusters in a simple way without code changes,” said Constellation Research Inc. analyst Holger Mueller. “This will make applications running on Kubernetes much easier to operate.”
Other improvements announced today include a new Errors Inbox for New Relic that serves as a single place for teams to view, triage and resolve errors across their application stacks. Available now, it should help developers to proactively fix errors before they affect application users’ experience, New Relic said.
There are improvements to network observability too, thanks to a new partnership with Kentik Inc. that’s available through an early-access program. That, the company said, should help DevOps and network teams collaborate more effectively to resolve any issues that might impact both.
One final addition to the New Relic platform is custom dashboard visualizations, which enable users to create customized views of their applications.
The updates came as New Relic said it’s launching a new Programmability Certification Program for developers to validate their expertise in extending New Relic One.
The company is also hoping to foster a wider community of users with the New Relic Student Edition of its platform. It’s being made available to students and teachers today and is free, with up to 500 gigabytes of telemetry data per user per month. The company, which also provides automated training and certifications for students, said the new edition will empower teachers to deliver more hands-on, practical lessons.
Asgar said New Relic’s goal with these new release is to provide observability for every engineer, regardless of who they are or where they work.
“We’re providing emerging startups, students, and teachers with exclusive discounts, credits, training and tools they need to enable data-driven engineering practices from the start,” he said. “By making observability tooling available to a wider range of current and aspiring engineers, New Relic is encouraging a more diverse developer ecosystem and a better overall future for the next generation of software builders and users.”
Image: New Relic
A message from John Furrier, co-founder of SiliconANGLE:
Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.
We really want to hear from you, and we’re looking forward to seeing you at the event and in theCUBE Club.