Today’s big story in the enterprise and infrastructure sector is, of course, the $2.5 billion acquisition by EMC of storage solution Isilon. The announcement had no sooner hit the wires when the long awaited announcement by Facebook of their public email system started to eclipse the biggest M&A story of the week in terms  of total ink devoted to the respective stories.

There is, at least from where I sit, an obvious common thread between the two stories, if you’re paying close attention.

From Techcrunch’s coverage today: “People are aware of Facebook’s Cassandra system, but now they’ve built something new called hBase,” wrote MG Siegler. MG Siegler incorrectly attributed the origins of the base technology of Facebook’s new email product to Mark Zuckerberg’s fifteen-person messaging team, though.

As our own Kristen Nicole pointed out today, though, hBase is a variant of Hadoop, a set of technologies the whole SiliconANGLE team explored in depth at Hadoop World earlier this year.

There’s a fat middle in the enterprise right now, when it comes to big data solutions, that’s slowly coming into focus. At one end of the spectrum, we see the “data temple” approach, as Wikibon’s Dave Vellante calls it. At the other end, we see giants like Google and Facebook rolling out “science project” solutions of their own that solve the problem of quick access to big data.

The Data Temple vs. Scaling Out

At Hadoop World, Mike Olson of Cloudera explained the problem of modern big data:

“Back in the day, if you had a data problem, if you needed to run business analytics, you wrote the biggest check you could to Sun Microsystems and you bought a great big single box, central server you could afford. If you had any money left over, you handed it to Oracle for database licenses. And you installed that database on that box and that was where you went for data. That was your Data Temple.”

“Data is growing faster than computers are getting bigger. Now. That means you can no longer have one box with all your data on it. It just isn’t possible for that to stay true. Data needs to be spread across many, many servers.”

“[Hadoop] was built by the biggest web properties on the planet—who ran into that problem maybe five years before the CNBC crowd—to solve that problem. Companies like Google, Facebook, and Yahoo had a lot of data spread across many servers and built Hadoop to coordinate and interoperate and run really powerful new analytics. This platform was built to solve the data and analytics problems that are confronting businesses today, not the ones that were the fodder for the relational market in the 80s and the 90s.”

In other words, it’s the classic juxtaposition of “scale out” versus “scale up.” Scaling up has long since become an impossible dream for many companies in the Web 2.0 sector, and we’re very quickly arriving at the point where it’s not possible for a large cross-section of the enterprise sector.

Hadoop and Scale-Out Storage: Servicing the ‘Fat Middle.’

It’s not difficult to imagine situations where everyday businesses and organizations can suddenly generate overwhelming mountains of data. At VMworld, Dave Vellante spoke to Isilon on #theCube, and in particular, they discussed the enormous data tracking problem that the Cincinnati public library faced, and how they used a scale-out solution to solve the problem.

The “fat middle” here consists of organizations that have need for storing and analyzing the vast array of data they’re constantly creating, but aren’t organizations like Google (who can write their own file system), or a Fortune 100 company that has money to blow on a Data Temple solution.

It’s the Fat Middle that typically turn to solutions companies like IBM, EMC and HP. Scaling out is a necessity, and it’s why IBM, EMC and HP have recently built or bought SONAS, Isilon and Ibrix respectively. Scaling out is a hot sector  of the storage market (one in which companies like BlueArc, DDN and startup Gluster are all players to varying degrees).

We spoke with Bank of America’s Abhi Mehta at Hadoop World, and he attested to the growing need for everyday companies to address their big-data problems.:

“You look at any single company, be it Bank of America, be it Walmart, be it Verizon, we’re all data companies at the heart of it—right?—we don’t push cash around—right?—it’s moving bits and bytes. And we realize that. We want to be good custodians of it and increase transparency.”

“I talk about the emerging business models and a concept that I call data-factories. We are witnessing the next industrial revolution, and it will be fueled by data. This revolution will be bigger than the industrial revolution because finally technology has democratized not just the access of data to a plethora of new companies, but also the ability to store, mine, clean, analyze, and produce data products to solve problems were previously unsolvable.”

I spoke this morning with Dave Vellante about this segment of the storage buying market, and whether he saw a need for Hadoop to be part of the solution.

“If I were Isilon, or any other the other players in the ‘scale-out’ space, I’d be saddling up to the Hadoop community by talking to Cloudera and working to build the hooks into my systems that would allow hBase and other related Hadoop systems to function on my infrastructure,” Dave told me.

The reasoning he gave behind his statement was that Hadoop is emerging as the defacto standard when trying to solve for big-data scale-out problems. It’s the only solution that has a truly organic community forming around it, and as we’ve seen today with Facebook, it’s become the go-to starting point when dealing with even the most basic of basic services: e-mail.

Forrester: “EMC Must Be Nimble and Responsive.” Isn’t That What This Acquisition Shows?

In his analysis of the acquisition today, Forrester researcher Andrew Reichman said: “In the case of the cloud, there has to be capacity somewhere, and Isilon is already in use as  a service provider back end, so this could turned to EMC’s advantage if they are nimble.”

This is exactly what EMC is trying to prove, through it’s recent raft of acquisitions. Since March, and even prior to that, EMC has been talking loudly about it’s ‘Journey to the Private Cloud,’ but more important than talking the talk, they’ve been walking the walk. In our talks with EMC’s president Joe Tucci, EMC’s COO Pat Gelsinger, and the laundry list of other EMC executives that have walked their way through #theCube, they’ve all had an acute awareness of not only the problem of big data, but an ambient awareness of the still nascent momentum of the Silicon Valley-spawned solutions to the problems posed by big data.

EMC’s Chuck Hollis quoted Pat today in his blog post about the Isilon acquisition:

“Pat Gelsinger (ex of Intel) is fond of saying that the rate of information growth appears to be outstripping the famous Moore’s Law of semiconductor evolution. Yes, there’s a tie-in with cloud.”

In short, EMC seems to be defeating the problem that most other tech giants seem historically plagued by: the inability to move quicker than startups and upstarts in solving the problems that no one had an inkling would ever be a problem a few years prior.

As John Furrier pointed out earlier today in his write-up of the acquisition, EMC is demonstrating leadership in a whole new dimension. We’ve been covering EMC and the other whales like a blanket over the last year or so, and in our view they get the market and are leading head and shoulders above everyone else.

We continue to roll-up more knowledge in this space with #theCube, but based upon the track record of innovation we’ve seen with EMC, I and the rest of us here at SiliconANGLE see EMC as clearly positioned to win.

[Editor’s Note: Photo credits: Robert Scoble, Michael Sean Wright and Ricky McGill. -mrh]