With less than two months left, 2018 is poised to go down in tech history as the coming of age for open-source software.

Need evidence? Over the past 10 months, notable open-source enterprises MuleSoft Inc., Magento Inc., GitHub Inc. and Red Hat Inc. have been purchased for a combined $50 billion.

Yet before jumping on the open-source bandwagon, observers would be wise to keep in mind that these technologies still depend on a sizable community of contributors to keep innovation fresh, and monetization of many open-source projects remains a struggle. So what is all the fuss about?

Behind the acquisition activity, there is a significant shift in dynamics taking place within enterprise computing. A new generation of developers is rethinking the compute infrastructure, taking full advantage of innovative distributed systems tools to power the network. It starts with the Kubernetes open-source container orchestration tool.

Container dominance across clouds, ecosystems

Kubernetes has reached an adoption point in the enterprise infrastructure where a biannual enterprise user survey by the Cloud Native Computing Foundation found that Kubernetes was the top choice for container management by 83 percent of respondents. Even more significantly, Kubernetes was running in private, public, hybrid and multicloud deployments.

The dominance of Kubernetes indicates that enterprises are embracing a management model that leverages the simplicity and agility of containerized microservices across the multicloud world. This is important because it has opened the door for developers, the new power brokers in enterprise computing, to share software more easily in network operations.

The result is a network infrastructure that becomes more efficient. By deploying containers onto a node, developers can use Kubernetes to manage application workloads across clouds while eliminating a lot of manual tasks. Developer software also runs more reliably when moved between environments. The new paradigm is “Build once, run everywhere.”

But what about big data, the fuel that powers enterprises today? An important byproduct of the industry’s wholesale move to Kubernetes is the growth of a new data ecosystem, built around the container technology, for supporting artificial intelligence, machine learning, and data management in cloud-native environments.

Examples of this growing data ecosystem are plentiful. In September, Nvidia Corp. announced its TensorRT inference server, a containerized microservice that enables AI-modeled applications within data center production environments. In August, IBM Corp. formed a partnership with MayaData Inc. to roll out a data-agility platform that improves deployment of stateful workloads on Kubernetes through an open-source container-attached storage project known as OpenEBS.

Despite the involvement of major players like IBM and Nvidia, it’s important to note that activity in this space is not limited solely to big tech firms. The French startup Dataiku Inc. recently released software that can generate a Docker image with AI code and automatically deploy it on a Kubernetes cluster. And Lightbend Inc. recently joined the Kubernetes microservices craze by launching the latest version of its data platform with the addition of the Kubernetes cluster orchestrator.

Despite all of the activity surrounding the Kubernetes world, there is still a need for a framework that could be used by developers to build data-driven applications in the cloud. In other words, DevOps for data.

Rockset emerges from stealth

To meet this need, a new analytics and search startup has moved out of stealth mode this month and announced a cloud service for running SQL on raw data. Rockset Inc. has built cloud-native technology for developers and data scientists to help speed data processing and accelerate application deployment.

The startup also announced $21.5 million in funding from Greylock Management Corp. and Sequoia Capital Operations LLC. “Through my investments in Docker and being on the board for Cloudera, I saw firsthand the rise of stateless apps, but not stateless databases,” said Jerry Chen (pictured, right), an investor with Greylock Partners.

Chen and Venkat Venkataramani (left), co-founder and chief executive officer of Rockset Inc., recently spoke with John Furrier (@furrier), host of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, at theCUBE’s studio in Palo Alto, California. “Every pitch I saw for two or three years tried to solve this data and state problem in the cloud by adding more boxes,” Chen explained. “It just got to be a mess.”

Rockset’s approach is to offer a search and analytics engine that is driven by cloud economics. Its managed cloud service eliminates the need for shards, indexes or server management, thereby freeing developers from the shackles of data infrastructure operations.

The firm’s technology leverages what it calls “fast SQL” with RESTful application programming interfaces, an architecture designed for networked applications, to deliver the solution. “If you know how to use REST APIs and if you know how to use SQL, you don’t need to think about hardware, anything about standing up servers, shards, re-indexing, resharding, none of that,” Venkataramani said. “You should be bottlenecked by your creativity and imagination, not by what my data infrastructure can do.”

Not surprisingly, Rockset is using Kubernetes as an integral part of its delivery system. “We’re huge fans of Kubernetes and Docker,” Venkataramani said. “The entire Rockset back end in built on top of that.”

Roots in Facebook

The presence of Venkataramani, along with co-founder Dhruba Borthakur, as the leaders of a new company in the cloud-native world is worth noting because of the backstory behind their careers. Both technologists were instrumental in the creation of the online search and data infrastructure for Facebook Inc. during the company’s rise to prominence more than 10 years ago.

When Venkataramani started at Facebook in 2007, the social media firm had 40 million monthly active users, according to the Rockset co-founder. “By the time I left, the systems were surveying 5-plus billion requests per second across 25-plus geographical clusters in half a dozen data centers,” Venkataramani recalled.

Venkataramani’s story points to an important element in the evolving saga of Kubernetes, open-source and the rise of cloud-native: Today’s networked systems are large, complex and time-consuming to manage. That has forced developers to seek out different tools they can use for processing, storing and delivering data across the enterprise.

Out of necessity comes innovation. This is why the open-source nature of Kubernetes has developed serious gravity. New tools for the container platform make a lot of other tools better, and the impact ripples across the technology universe.

Looking toward the future

What does the future hold for a technology that has emerged as one of the most significant advances over the past 10 years? Brendan Burns, distinguished engineer at Microsoft Corp. and a co-founder of Kubernetes, posted a lengthy discussion of this topic several months ago and summed up the future in one word: serverless.

There are already signs that Kubernetes will move in a serverless direction, where the cloud provider dynamically manages server resources. Microsoft announced a serverless container offering, Azure Container Instances, in 2017. That was soon followed by the introduction of Amazon Web Services’ Fargate a few months later. In July, Google joined the party with its own Kubernetes-based platform, Knative, designed to deploy serverless workloads.

Concurrent with the introduction of Kubernetes-based serverless platforms by the major cloud providers has been development within the open-source Virtual Kubelet project. It’s important work, as Burns pointed out, to bridge the gaps between the Kubernetes application programming interface and serverless containers.

“The integration of the Kubernetes orchestration layer and serverless container infrastructure is crucial to the future success of both Kubernetes and the serverless infrastructure,” Burns said.

At the core of the cloud-native movement is a fundamental belief that a rising tide will lift all boats. Better technology in the cloud means better technology, period.

“We’re not optimized to sell latte machines. We’re selling coffee by the cup,” Venkataramani said. “We want to put it in the hands of as many people as possible and make sure we are useful to them.”

Here’s the entire video interview with Venkataramani and Chen, one of many CUBE Conversations from SiliconANGLE and theCUBE:

Photo: SiliconANGLE