On track one, we have complicated technologies for hot new trends such as big data analytics and artificial intelligence. On track two are the growing number of tools that collapse these complexities into simpler abstractions — things like cloud services and serverless computing. The race is neck-and-neck, since no sooner is something simplified than something in its evolutionary path pops up to complicate things once they begin to scale.

Simplification is about to win a round against complexity, in large part thanks to emerging automation techniques in enterprise computing. Mesosphere Inc. is taking a whack at the notoriously busy Kubernetes container orchestration platform. The open-source Kubernetes system automates the deployment and scaling of containerized apps.

Founded in 2013, the company bases its data center operating system, known as Mesosphere DC/OS, on the open-source Apache Mesos project. Mesos virtualizes the dickens out of whole data centers at huge scale and renders them to users, essentially, as a single computer to manage.

It began as an experiment “to figure out if you had to rebuild Google’s proprietary architecture for hyperscaler computing, what might that look like,” said Ed Hsu (pictured), vice president of product and product marketing at Mesosphere.

Hsu spoke with Lisa Martin (@LuccaZara), co-host of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, and guest host John Troyer (@jtroyer), chief reckoner at TechReckoning, during this summer’s DockerCon event in San Francisco. They discussed the origins of the Mesos open-source project and how Mesosphere determines product build-outs, including the newest Kubernetes and Jupyter releases. 

This week, theCUBE spotlights Ed Hsu and Mesosphere in our Startup of the Week feature.

Simplifying Kubernetes integration

The likes of Twitter Inc. and Airbnb Inc. use Mesos to solve otherwise insurmountable scale and performance obstacles, according to Hsu. “Arguably, without Mesos as a technology, I don’t think Twitter would probably exist today,” he said, recalling the frequent crashing that once plagued the site.

Mesosphere’s integrated platform for data and containers automates roll-out and production operations for containers and data services. It has just added new features to deliver Kubernetes and Jupyter Notebooks as a service. This is a first for many frustrated developers who love the idea of Kubernetes but loath the assembly work.

“We basically enable you to very easily adopt all types of new cloud-native technologies like Docker, Kubernetes, Spark — all the things that you think about to build these world-changing applications,” Hsu said.

Mesosphere’s automation of Kubernetes clusters operate consistently across hybrid cloud environments. The company’s platform resides below a container orchestrator. “Mesosphere enables customers to really get one turnkey installation and operation of that technology,” Hsu stated.

Forty-seven percent of information technology leaders plan to deploy containers in a production environment, according to Diamanti Inc.’s 2018 “Container Adoption Benchmark Survey.” The way the word containers echoes across the tech scene, one might expect to see a very high percentage of existing deployments in the enterprise. However, only another 12 percent of those surveyed said they have already deployed them.

Relatively modest enterprise adoption is likely due to the problem of managing large numbers of containers in production. Many companies are struggling to get their hybrid-cloud or multicloud environments in order, managing swarms of containerized application across a messy IT landscape.

“Although there is growing interest and rapid adoption of containers, running them in production requires a steep learning curve due to technology immaturity and lack of operational know-how,” stated Arun Chandrasekaran, research vice president at Gartner Inc.

A trunk full of junk

The conversations at Kubernetes and Cloud-Native tech conferences in the past year or two have deliberated the challenges in building up the ancillary technologies to appropriately optimize Kubernetes. “It takes a spectrum,” Clayton Coleman told theCUBE at KubeCon + CloudNativeCon in Austin, Texas, last December. “This is where an ecosystem really comes into play,” he said.

The Cloud Native Computing Foundation, the new home base for Kubernetes after Google handed over its creation to the open-source community, has made efforts to flesh out standards and best practices.

“We all need to work together to make those standard solutions happen,” Coleman said. For example, there is the CNCF conformance profile for Kubernetes. “It’s a new effort that intends to take the tests that we use in Kubernetes to make sure that it’s working correctly and use that to say, ‘This is something that you can rely on every Kubernetes distribution also supporting,’” Coleman concluded.

With a burgeoning ecosystem comes opportunities for startups, such as Istio, each tackling one or another kink to smooth over large-scale Kubernetes deployments.

As-a-service at your service

For end users looking for efficiency integrations, the Kubernetes as a service offering that Mesosphere is bringing to the table is probably more appetizing.

“There are many different companies that provide individual point solutions for those technologies, but how do you bring it all together in a multi-tenant way?” Hsu asked.

Mesosphere pools the underlying resources that run technologies such as Kubernetes (or Apache Spark or Cassandra database) the way that VMware Inc. pooled servers into virtualized resources, Hsu explained.

“So the cost savings you saw from server consolidation, we’re doing from cluster consolidation,” he said. This dramatically reduces costs, automates operations, and allows business units to share an infrastructure while maintaining resource guarantees.

“Instead of getting a whole team to figure out how to run stuff, it’s literally one-click installation or a single command on the DC/OS command console,” he said.

Companies can now “run all these different types of data services in a highly pooled way so that you don’t have different clusters that are turning into  snowflakes that cannot be reused by other teams,” Hsu concluded.

Watch the complete video interview below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of the DockerCon event.

Photo: SiliconANGLE