Oracle Corp. faces severe challenges from Big Data on three major fronts: It will see a steady erosion of its income as the market moves to Big Data technologies that Wikibon estimates cost a third of traditional RDBMS technology. Second, Oracle needs to move up the stack to survive in the Big Data economy, as the core database business moves to Hadoop and associated open source technologies. Third, its existing RDBMS database, the center of its product line, is aging technology that will struggle to meet the challenges of Big Data.

On the plus side, Oracle has important strengths. The first is time. While the move to Big Data is inevitable, it will not be fast. The Hadoop stack is still immature and difficult, requiring new skills both in the IT shop and the business. That means the shift in Oracle’s traditional business will be gradual. Oracle can continue to generate strong revenues in its core product line for several years, giving it financial resources to develop the new set of products it will need for the Big Data future.

Second, it has a highly integrated stack. Wikibon CEO David Vellante recently published a detailed analysis of what he termed the “Amazon economy”, the cloud-service based competition to traditional on-premise IT infrastructure that is rocking the industry. In it, he says Amazon competitors need massive scale or very tight integration, and he uses Oracle as his example of that second strategy.

Oracle’s problem is that, at present, its stack is integrated around its RDBMS database, and if it is to move into the Big Data future it will need to either make major modifications to the core technology, supplement it with Hadoop and other technologies,\ or replace it altogether.

Third, Oracle has a strong research and development culture, a fact that was underscored by founder Larry Ellison’s decision to step down from the CEO role to run Oracle development. Oracle has always been a determined competitor. It will need that fighting spirit to navigate this transition.

New Vision

.

However, Oracle needs to refocus on a new vision of itself in the radically different IT market of the future. It can buy companies but not vision, and without it, it risks the fate of other one-time leaders such as Digital Equipment Corp. (DEC), which ignored major technology and market shifts until it was too late.

Oracle can, and probably will buy and integrate Big Data startups to build a portfolio. What it cannot buy is Hadoop, MapR and the other parts of the open source stack that are the base database and analysis technologies for Big Data, the equivalent in the Big Data stack of the Oracle RDBMS, on which all of Oracle’s present applications run. And while Oracle as a NoSQL database, it cannot buy a corner on the NoSQL database market, Big Data platforms like Splunk and unique cognitive computing platforms like IBM’s Watson.

Even with a full Big Data stack, Oracle will not be able to realize anything close to today’s Red Stack revenues. The combined Hadoop-NoSQL software markets will grow to $3.4 billion in annual revenues by 2017 according to the Wikibon Big Data Revenue and Market Forecast 2013-2017. By comparison, says study author and Wikibon lead Big Data Analyst Jeff Kelly, the combined RDBMS/data warehouse market today is nearly 10 times bigger, with a big chunk of that generated by the Oracle Red Stack. This means that margins in the Big Data market will be a fraction of those Oracle enjoys today. Oracle can make up some of the difference in volume, but the combination of open source bottom layers of the Big Data stack, much lower margins, and the inability to dominate the market will probably equate to much lower profits overall for Oracle.

Will it happen?

For all their complaints about licensing fees, Oracle customers are not moving off the Red Stack anytime soon. It just works, and those applications are core to the business. Oracle financials, for instance, are the gold standard definition of compliance with increasingly demanding business regulations. A lot of Oracle’s R&D is focused on keeping them that way. If Oracle can maintain its present Red Stack business and develop Big Data products to provide growth, it has a chance of successfully migrating to a new business model going forward.

The problem is that IT markets are being disrupted piece-by-piece by new age cloud applications. Salesforce.com grabbed a huge percentage of the CRM market that was once dominated by Oracle’s Siebel technology. SaaS vendor Workday is disrupting the HR market and to a lesser extent so financials, growing 70 percent a year in the process. Some customers are already moving off on-premise RDBMS-based financial systems. And Workday foresees a day around 2020 when many companies will have their HR and financials combined on its cloud platform. That is a clear threat to Oracle’s core business.

The open question is what part of the IT software market will be disrupted next. It is only a matter of time before the next generation of ERP moves to the cloud. Oracle could respond by attempting to acquire Workday and other competitors, but piecemeal acquisitions are difficult to stitch into a coherent whole and the margins of cloud startups are much lower than Oracle’s traditionally have been.

Flexibility

.

The change is already happening. At the recent HP Vertica, Splunk .conf2014 and BigData NYC conferences, Wikibon Project Chief Analyst David Vellante interviewed several Oracle customers who are capping their RDBMS environments and shifting new investments to Hadoop and other Big Data technologies. Some are justifying initial Big Data installations based upon measurable ROI they will gain from moving some applications from Oracle ot open source technologies.

Some of the most radical stories from practitioners interviewed on theCUBE came at the HP Vertica Conference in August. Jordan Cherney, systems architect with furniture and home furnishings company Wayfair.com, said  his company originally used the IBM Netezza appliance to support its business analysis. As volume grew, it replaced Netezza with Vertica running on top of Hadoop and realized better analysis in real time and more flexibility because it eliminatee the need to impose a data schema.

Blue Cross/Blue Shield Association in Chicago replaced 37 separate data warehouses and all three leading relational databases supporting its 37 member insurers with a single Hadoop database also using HP Vertica for real-time analysis.  Janath Manohararaj, team lead for database services, said  that under the old architecture”data was everywhere” with members making multiple copies of their databases. Hadoop and Vertica allowed the organization to establish control over the data, centralize its system, provide much faster data ingestion and analysis and save a great deal of cost in licensing fees and day-to-day operations.

Cardlytics, a cloud provider of customer analytics for banks worldwide, moved from an RDBMS analytics platform to Hadoop and Vertica “when we signed our first large bank,” said Craig Snodgrass, chief strategy officer for the financial services company. The company discovered that it simply could not ingest the data fast enough with the RDBMS to provide the real-time analysis that is the core of its service.

The big problem that drove these companies out of the centralized data warehouse was that RDBMS technology requires that all data be organized according to a data schema, which defines all the questions that can ever be asked of the database. This is what made it impossible for their data warehouses to ingest the data at the rate required to produce the near real-time analytics that they needed. Schemas also makes RDBMS inherently inflexible, unable to accommodate new ideas and changes in markets that require answers to new questions. And they can create huge complexity and expense in every aspect of data management.

Big Data technologies, in contrast, do not use a data schema on ingestion. The schema is applied when the data is analyzed and reflects the questions asked in that analysis. Multiple schemas can be applied to the same data to answer different questions.

New relationship

.

This changes the relationship between the analysts and the data, said Greg Hrebek, director of engineering for New York Air Brake, in an interview on theCUBE at Splunk .conf2014. The old relationship was like getting married. “Everything was well defined, and the longer you were together the harder it was to get divorced,” Hrebek said. The new one is more like speed dating. “I can try different things out without a commitment. That dramatically accelerated our pace and ability to service our customers.”

This 124-year-old manufacturer of brake systems for class A railroads like Union Pacific and CSX uses Splunk to capture and analyze large amounts of operational data from its customers’ road engines to provide valuable insights. For example, it provides its clients with strategies for maximizing fuel efficiency on their freight routes to typically achieve a five-percent to eight-percent  savings in diesel fuel. Since a class A railroad burns more diesel fuel in a year than the entire U.S. Navy, this can translate into hundreds of millions of dollars in savings.

New York Air Brake analyzes engine performance data to determine when engines need service, allowing them to stay on the rails longer between service intervals. The analysis saves customers an average of $1 billion each, Hrebek said. Those railroads had all the same data, but none were able to do the analysis using their RDBMS data warehouses.

So what can Oracle do? It can start by embracing Big Data internally. Wikibon believes that the big winners in Big Data will be the practitioners who, like New York Air Brake, transform their businesses. While all the traditional IT vendors face a difficult evolution to the new market realities, Oracle, Teradata and SAP are particularly vulnerable because their entire businesses are based on RDBMS technology. Oracle needs a transformation while it still has the resources to transform, and Big Data, ironically, is the key to its future.