Geoffrey Moore of Mohr Davidow Ventures and author of many books including Crossing the Chasm, Inside the Tornado, and Inside the Fault Line, delivered the keynote at the Hadoop Summit 2012. Moore also took time to talk with John Furrier, Founder of SiliconAngle, and Jeff Kelly, Analyst with Wikibon, as part of theCube’s conference coverage.

The Importance of Big Data

“We are putting the entire world on the Web and have digitized the consumer experience,” Moore said. “We are now digitizing the business experience including government and healthcare. The toolset you need to digitize requires Big Data because it’s not just about handling data before and after, it’s about handling data during transactions. Without Big Data, you are blind and deaf and in the middle of a freeway.”

Moore also discussed how early media companies, such as Yahoo, which found the Hadoop stack, found the Big Data space, but now the rest of the world is figuring out how and when to start working with Big Data. “It will not be mono-directional,” Moore said. “When you have a technology disruption like Big Data within a human-social context, the two interoperate with each other. Humans will slow down some of the technology because they can’t handle it all at once.”

In assessing how the market will evolve, Moore said early adopters will start with projects. Those that are successful will get visibility while others will hold off. But as successful use-cases emerge, others will jump in. “The markets will emerge around use-cases but we are still early in the market and there’s still a lot of work to do,” said Moore.

Keys to Big Data Success

Business should be aware that Big Data is evolving on a bigger campus at a faster pace than many past technology innovations. Businesses thus need to adapt their business model to the adoption lifecycle. “You may not be strong at every step of the lifecycle, but that’s OK,” Moore said. “The people who will be stellar at the beginning are the technologists who take on the projects. Once Big Data crosses the chasm and becomes mature, the focus will move from creativity to scalability and reliability.”

Moore predicts the market will look for use-case domain expertise. Even though the technology is new, application vendors are leading the parade. This will continue until the market is ready for horizontal applications, which will then be adopted vertically. Every industry will eventually adopt Big Data Moore stated.

BYOD Impact

“The demand of BYOD is coming from the top, such as the CEO, so IT can’t say no,” Moore said. “IT typically puts a special team on it and surrounds the CEO with services to make sure his or her personal device works right. Then the CEO says, ‘great, now I want it for my entire sales force.’ IT then realizes it does not have resources. This puts a lot of pressure on CIOs when it comes to infrastructure, wireless access, authentication and security.”

Is There a Chasm to Cross?

“Crossing the chasm is a great B2B model, so for Hadoop it’s perfect,” Moore said. “But for businesses spinning up on the Web, it’s more like a tornado. You either get one going or you will fail. You need to get a million users in a short amount of time or you become stale.”

While talking about launching Big Data initiatives, Moore said it takes visionary leadership to move a big part of the budget. “It’s usually not the CIO, but this person and the CIO will then have to bond at the hip,” Moore pointed out. “If that person does not exist, it’s a mistake to do an early market project. Instead, do some pilots or testing. You need to ‘vaccinate’ your IT team to give them experience at handling this new ‘irritant.”

Smartphones Change the Market Dynamic

Smartphones and other personal devices have already changed the consumer and media markets as well as the political market with its ability to spread ideas. Moore thinks it’s changing the sports-fan experience too and that education and healthcare could benefit, but they tend to be conservative in making changes and have many constituencies to bring on board.

Moore finished by saying, “There’s a sea change: digital was a record keeper but now digital is the world. We will need to wire the entire nervous system, and the Internet is the spinal column. We now we need to extend it outward—that’s Big Data and analytics. It’s an infrastructure problem that won’t be an overnight success, but the key question is, can you be part of the nervous system?

Moore added that the current incumbents don’t have to innovate, but they do have to adapt. “There’s a lot of money in old business models that business are afraid they’ll lose if they embrace a new model too early. But if you don’t, your customers will eventually say you are irrelevant. It’s a dance—if you make reasonable progress, customers will put up with you walking instead of running.”