Last October I wrote:

“By late 2012, federated storage will be the architecture of choice for large new storage deployments. These capabilities will dramatically improve IT’s ability to respond to business needs with minimal disruptions. IT organizations should plan to aggressively adopt federated storage as it becomes commercially available.”

Around that same timeframe, Nick Allen wrote a piece about Chad Sakac’s Long Distance Live vMotion demo at VMworld 2009 and asked :

“So, has EMC solved the long-distance cache coherency and distributed lock management problem that has plagued the industry forever?”

Today (3/11/2010), EMC’s President of Information Infrastructure, Pat Gelsinger unveiled a vision for federated storage which makes our two posts last year pretty timely. You really have to hand it to EMC. In late 2007 things were getting kind of boring and Joe Tucci leaks Hulk and Maui. Then a few months later, EMC lands a haymaker by announcing flash drives in Symmetrix. Next was the buzz EMC created around its private cloud vision; followed by Fully Automated Storage Tiering (FAST).  And now– virtualized, private cloud-ready, dynamically scalable, distributed cache coherent, automated, always-on, efficient, on-demand, active/active, globally federated storage. (Thanks to Chuck Hollis for providing Gelsinger’s slides which enabled that sentence).

What is Federated Storage?

Here’s Wikibon’s definition:

Federated storage is the collection of autonomous storage resources governed by a common management system that provides rules about how data is stored, managed, and migrated throughout the storage network. In this definition, storage resources include disk capacity managed by controllers or appliances controlling multiple arrays.

Practitioners should think of individual resources (e.g. arrays) as nodes within the federation.

Shades of Global Sysplex

Global Sysplex was an IBM mainframe innovation with the following attributes:

• A geographically dispersed set of processors within synchronous distances.
• Separate resources acting as a single global pool.
• Applications and data could be moved around and shared across processors.
• One key ingredient of Sysplex was the Sysplex timing mechanism which acted as a time stamp (not as important for federated storage).
• Another element was global cache management to store things in a global shared cache (important for federated storage).

By enabling a loosely coupled set of storage resource nodes to act unilaterally and still be managed centrally, organizations can create networks of virtually limitless capacities, move data and applications globally, eliminate disruptive migrations and dramatically improve recovery.

Oh…and this time you don’t need a bazillion dollar mainframe complex to do it.

What Problems Does this Solve?

Today’s remote replication systems (e.g. SRDF) can be incredibly complex and expensive. Organizations need to set up disk systems at multiple locations and pre-allocate space and communications lines and essentially purpose build replica systems to support specific applications. Federated storage enables a general purpose virtualized pool that can be established to migrate workloads, free up disk space, end of life assets and bring on new resources much more quickly and at substantially lower costs. Planning is simplified and RPO is improved.

The big benefits will be at synchronous distances but federated storage will also help improve the efficiencies at asynchronous distance. I wouldn’t hold your breath waiting for asynch active/active (remember Chad’s demo took a long time to move a few GBs – how long to move a TB?) but the overheads of wait times and acknowledging writes will be dramatically improved by avoiding time-consuming remote gets and message queues.

And the services delivered can be way more leveraged across the application portfolio.

Questions and Comments from The Blogosphere

@VanessaAlvarez1 tweeted:

“#EMCVS first version w/b appliance, why do we always start with a HW piece? Why not just go straight to making it a feature?”

Gelsinger mentioned that one of the secret ingredients was IP acquired from Yotta Yotta. @RayLucchesi wrote on his blog today:

“Yotta Yotta’s product was a distributed, coherent caching appliance that had FC front end ports, an Infiniband appliance internal network and both FC and WAN backend links.”

My understanding is this isn’t just Yotta Yotta re-purposed but that IP was the starting point.

Ray also said:

“So it looks like #emcvs is solving the DaaD problem. makes active-active and vmotion ovr dist easier but is their a mkt for it?”

I think so. First, most organizations in the F500 doing SRDF will look at this over time and see it as a more attractive alternative. But I think this vision has broader appeal than just the F500 because it can potentially scale down, over time.

The best comment of the day on Twitter goes to @storageio (Greg Schultz).

“Given Yotta Yottas history of delivering Nada Nada, can EMC turn it around with #emcvs into Lotta Lotta?”

I have no idea but that’s pretty funny.

{Editors Note: Please welcome Dave Vellante as a new contributor to SiliconAngle.  Dave is a seasoned vet in the computer business who spend 15 years at IDC as a top analyst and now entrepreneur and founder of Wikibon.  Wikibon is a new open analyst firm based out of Boston and Hopkinton Massachusetts.  Wikibon has become the Web’s first and largest technology research and advisory organization where peers help each other make better technology investment decisions and share best practices. Dave is active on Twitter @dvellante and a great analyst  – Welcome aboard Dave! – Cheers John Furrier, @furrier}