Blog

The Cloud Isn’t a Mainframe. Seriously.

October 23, 2009
Filed Under: in Analysis, Cloud Collision, Featured Articles, Infrastructure 2.0
Author: James Watters

Welcome back.

Warning: the following is a good-spirited rant.

The "return to the mainframe model" analogy for cloud computing is utterly simplistic in its thinking- it makes for a nice "big animal picture" for the masses, and makes veteran IT professionals feel like they've seen this all before. In reality, the analogy makes us dumber about how to maximize the cloud computing trend, because its simplicity belies the two biggest trends in the cloud: over abundance of compute resources and a fragmentation of processor to processor coherency or machine state.

The differences are stark, the similarities are thin…

…the clear contrast: scarcity vs. abundance of computing resources-

The mainframe / terminal model was founded on a fundamental scarcity of compute resources. At the time, computers were so large that they required highly specialized rooms, HVAC, operators- the works. If you wanted to use them you often stood in line, made your case, and were eventually rewarded with some of the scarce and highly valuable cycles via a terminal.

The cloud computing revolution, quite the opposite, is founded on a fundamental abundance of powerful but disjointed computing. There are so many servers sitting on networks all over the world that they are now competing for your attention. The trick is assembling the software, processes, and applications suitable for taming the millions of public web connected server nodes into something cohesive.

Mainframes are designed at both the hardware and software level to maintain a high degree of state-fulness and transactional capability; the current wave of cloud computing is being built around task queuing and more loosely coupled sets of services. This is a very real difference. I recently sat in a cloud computing breakout session at Zendcon where a senior developer was desperately asking for help from the other attendees, saying: "We've been looking at this hard, and porting our systems to non-relational databases in the cloud is going to be hard... we need help."

Unlike the mainframe era, the distributed systems of the public cloud might not be the right home for all of your workloads just yet. Cloud computing is not a systematic catch-all; instead it is a disruptive 'and'.

The only real connection is the one of 'remote trust'-- using a non-proximal resource. The exact same simplistic thinking now has even the freaking Register calling the Nokia SideKick incident a failure of the cloud storage provider.

But let's examine this whole 'trust' angle a moment...

'Trusting' the mainframe operators wasn't a choice; it was a necessity. You couldn't afford to have your own computer, plain and simple. And let's not forget the simplicity of networking, security, etc. in those days-- there was no such thing as BGP, because for most of the mainframe era, 'router' wasn't even a word yet. So even if you push the 'trust model' and remote access analogy, I will counter that there was no choice but to trust, only necessity; and by remote I hope you are keeping in mind the comparatively simple network topology connecting the user to the resource.

Scarcity vs. abundance, coherent vs. coupled are just as important.

How did we get here anyways? It's no accident Google and Amazon were early players in the cloud computing market. They rode an incredible shift in consumer behaviors, from offline purchases and decision making to increasingly online purchases and decision making. To solve their massive, parallel demand increases, they had to acquire hundreds of thousands of servers. They had to focus on the SW and operational processes to meet a tsunami of demand. Similar to the the terminal access driven era of the 70's, they then offered up a slice of this unified uber-resource for individuals to consume - I get that part--but the differences in the why (abundance) and the how (loosely coupled and stateless) are more strategic to the future of the market than the similarity.

“The victory of SW over HW”

The mainframe model of programming wasn't nearly as abstracted as our modern model for a simple reason--you just couldn't waste a MIP. HW was king, and thus the cult of the all-mighty server was born. We tolerated incredibly long procurement cycles for servers because of this "all-mighty server" legacy we inherited from the past. Today's reality is the opposite. In our future servers will be procured in minutes or less--the worship of scarce HW is over, and all eyes are on the processes and software that allows us to use the abundant buzzing swarm of compute.

We now have so many powerful computers that we are willing to greatly sacrifice raw performance for manageability. This morning on the "CloudCamp in The Clouds" webcast I asked a question about Hadoop's suitability for public cloud (AWS) usage. The answers were pretty interesting. The first answer was "oh don't do that, Hadoop is so disk I/O intensive that it's a waste to do it in a highly virtualized environment." The second answer, from meeting organizer @ruv, was more interesting: "Who cares? You have so much compute out there, its not about raw performance but simplicity and convenience."

So What…

- Let's be specific, not general in our thought models whenever possible. I wish we had an all-powerful SMP server in the sky we could trust with every workload with no need for any local processing, but that's just not where we are or where we are going. Check out the memory usage of Seesmic desktop sometime - mine is routinely in the 1/2 gig range.

- The mainframe metaphor is more nostalgic than informative, and leads to some of the public misunderstanding of 'the cloud'. It may be fine for a CNN viewer, but beyond that, its utility is dubious to me.

- I know that many of these points are obvious, and that not everyone is using this metaphor, but I see enough people doing it that I wanted to explore it for a few days as a thought model and report back.