The fifth annual Hadoop Summit in San Jose has come to a close. As one would expect, theCube was streaming live coverage and analysis directly from the conference, hosted by SiliconANGLE founder John Furrier and Wikibon analyst Jeff Kelly. Often called the “ESPN of tech”, theCube was onsite, extracting the big data signals from the conference noise.

The Hadoop Summit is definitely a conference geared toward technology-seeking individuals, comprised of a number of technical sessions with hints of business cases and use cases trickling throughout. A successful event on many fronts, Furrier gave this year’s conference a score of 8.5 points out of 10, namely for being sold out, filling the sessions with plenty of attendees, enabling great conversations throughout the conference, and creating good vibes overall. “We didn’t experience a lobbycon,” said Furrier. “People weren’t trying to sell anything – though the attendees were keen to move beyond the detail of the technology to discuss the market opportunity. We are starting to hear discussions about the real business cases being built around big data.”

Furrier and Kelly were able to speak with a few companies during the conference that were happy to share their own business/use cases with the SiliconANGLE audience. Klout VP of Engineering David Mariani, for example, described Klout’s, which are built on a combination of Hadoop and big data analytics, allowing this social influence company to both process and store its big data cost-effectively. Mariani also noted that Klout would like to see some consolidation between these two functions of infrastructure and analysis. “We still need schemas on the unstructured data in order to get the most out of it,” Mariani said.

Analytics was undoubtedly a hot topic throughout the entire conference; companies are looking for the best ways to extract the analytics from the databases. And, when we start talking about schemas and other SQL-like features, “the issue of scale quickly surfaces,” said Kelly. “Bringing big data to the masses means making it comfortable for the end user. To address these issues, we will likely see a consolidated project where the analytics reside directly within the platform.”

Speaking of consolidation, we can certainly expect to see quite a bit of such activity in this big data space over the next few years. We have already seen some small business consolidation taking place, and we will likely watch the traditional market lifecycle play out as the usual giants like IBM, HP, Microsoft, etc., sit back, observe, and wait for the right time to pounce, making acquisitions and taking advantage of a hot technologies or even great business deals.

Of course, these startup companies that are building apps on top of the infrastructure are not stupid. They know that an acquisition is a completely viable exit. And, as these companies posture for positions as up-and-coming leaders in the big data space, we could see trillions of dollars changing transitioning between vendors as companies try new ways to get the most out of their data. The large players in the traditional database space, (the incumbents, if you will) that are waiting it out on the sidelines will need to pay very close attention to how things proceed if they are to keep their competitors’ grubby paws of their businesses and revenues. “Companies will find they need to play both chess and poker at the same time,” said Furrier. We’ll see which can manage their moves and their hands both strategically and tactically over the next few months and quarters.

Consolidation is a natural phase within the market development lifecycle. While the big data landscape may be larger out of the gate and moving more quickly than businesses conceived 10-15 years ago, during an interview in theCube, author and Mohr Davidow venture partner Geoffrey Moore reiterates the same messages presented in his best-selling books, moving from technology discussions, to business cases, to consolidation as a natural progression for pretty much any tech market. “The first use cases are driven by huge pain where the outcome clearly outweighs the risk,” said Moore. “If the use case is repeatable, a market is developed and one or more businesses are formed. As more exceptional use cases are developed, thereby creating one or more sub-markets, the market begins to cross the chasm.

“To succeed, companies will need to adapt their business models to the lifecycle in order to be successful,” added Moore. “In this particular space, the application vendors will lead the parade, getting the infrastructure adopted within specific verticals for specific use cases. Then, after these target verticals cross the chasm, the horizontal players (IBM, HP, Dell, etc.) will come in and take it out.”

Needless to say, this all points to a booming ecosystem in which the developer will play a key role. With a new and exciting infrastructure at the core, where the ecosystem will be driven by a number of developers creating new technologies (apps) that sit on top of the infrastructure, we are already seeing tremendous entrepreneurial spirit creating a lot of startups. The excitement has only begun.