UPDATED 08:00 EDT / FEBRUARY 24 2015

Paradigm shift: How Apache Mesos is reshaping the data center into a giant PC

hp-servers-in-apple-data-centerMost people couldn’t get much done on their computers without an operating system. Working the hardware directly would require the specialized knowledge and near-infinite patience to manually feed instructions to each processor on the motherboard in its specific language for every single action.

It’s an impossible proposition. Yet the infrastructure professionals who keep the world’s data centers humming along manage a comparable feat on an incomparably larger scale as a matter of daily routine. They determine workload requirements, allocate the necessary resources and perform all the other activates by hand that are usually executed automatically on a personal computer.

It doesn’t have to be that way, or at least not according to the backers of an emerging open-source project called Apache Mesos that promises to bridge the chasm between the consumer and enterprise worlds. The framework offers to provide an operating system for the data center.

The concept is a radical departure from the current reality of manual management that traces its roots back to the mainframe era. Mesos ofers a fresh approach that takes a page directly from world’s largest data center operators.

The platform was originally developed at UC Berkeley’s famed AMPLab, the birthplace of pioneering open-source technologies like Apache Spark and GraphX, with direct input from Google. The search giant was working on a new system to manage its data centers at the time and took the opportunity to exchange key lessons with the creators of Mesos.

After its release, the open-source framework went on to find a home at other web giants such as Twitter, Inc. and Airbag, which funneled the lessons they gleaned from running large-scale production implementations back into the upstream initiative. The end result is a platform that not only provides the conveniences of an operating system but acts like one, too.

Mesos elevates distributed workloads to the same level at which traditional single-tenant applications execute on a single computer. The complex details of the underlying hardware are hidden away under a programmatic interface that exposes – or more accurately, offers – resources for consumption on an as-needed basis.

Services, which are deployed in containers to provide the scale-out equivalent of process isolation, determine how much capacity to accept through a built-in scheduler that can be optimized for the specific requirements of the workload. That two-tier approach makes Mesos more flexible than other clustering technologies that make the choice independently of the application and puts it an entire world apart from how most organizations provision their infrastructure today.

Business processes are typically deployed in virtual machines that are complicated to manage and immensely difficult to re size, which leads to a situation in which there are almost always too few or too many resources allocated to an instance. Organizations didn’t have much choice but to accept that situation until now.

While Mesos addresses that trade-off, the operating system analogy doesn’t fully hold for a data center with thousands of computers. A container that ends up on a server with less bandwidth than the other nodes in a cluster, for instance, will inevitably create a bottleneck regardless of how many layers of separation there are between the hardware and the administrator. Some problems will always require moving down the stack, just as not every workload can be effectively accommodated in a fully abstracted environment.

Mesosphere, the startup that members of the original AMPLab team created to commercialize the project, has been working to fill the gaps with its commercial version of the platform. Unveiled in December, the distribution combines the framework with a number of other components from the open-source ecosystem to address key needs such as the ability to effectively scale services. But the question whether the enterprise is truly ready for a data center operating system still looms large.


A message from John Furrier, co-founder of SiliconANGLE:

Support our open free content by sharing and engaging with our content and community.

Join theCUBE Alumni Trust Network

Where Technology Leaders Connect, Share Intelligence & Create Opportunities

11.4k+  
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+ 
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.

SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.