In 2015, data agility will emerge as a top focus, organizations will move from data lakes to processing data platforms, and self-service Big Data will go mainstream. This is all according to John Schroeder, CEO and co-founder of MapR Technologies, Inc., a provider of an enterprise-grade distribution for Apache Hadoop.

Schroeder’s predictions about Big Data are all part of our second annual Technology Predictions series in which industry experts share their predictions with us about the hot tech trends that they think will take center stage in 2015. Read on for more from Schroeder.

.

.

2015 will be the year in which organizations will move Big Data deployments beyond initial batch implementations into real time. This will be driven by the realization of the huge strides that existing industry leaders and soon-to-be new leaders have already made by incorporating new Big Data platforms into their operations, integrating analytics with “in-flight” data to impact business as it happens. Read on for more of my predictions for 2015.

.

Prediction No. 1: Organizations will move from data lakes to processing data platforms

During the past year, data lakes and data hubs have represented a popular first deployment for Hadoop. A data lake or a data hub is a scalable infrastructure that is both economically attractive (reduced per-terabyte cost) and agile; it has the ability to store various forms of both structured and unstructured data. The ability to use thousands of servers and store petabytes of data at less than $1,000 per terabyte per year is a core benefit of Hadoop.

In 2015, data lakes will evolve as organizations move from batch to real-time processing and integrate file-based Hadoop and database engines into their large-scale processing platforms. In other words, it won’t be about large-scale storage in a data lake to support bigger queries and reports. The big trend in 2015 will be around the continuous access and processing of events and data in real time to gain constant awareness and take immediate action.

.

Prediction No 2: Data agility will emerge as a top focus

Legacy databases and data warehouses are so expensive that database administrator (DBA) resources are required to flatten, summarize and fully structure the data. Upfront DBA costs delay access to new data sources and the rigid structure is very difficult to alter over time. The net result is that legacy databases are not agile enough to meet the needs of most organizations today. Initial Big Data projects focused on the storage of target data sources.

Rather than focusing on how much data is being managed, in 2015 organizations will focus on measuring data agility. How does the ability to process and analyze data impact operations? How quickly can they adjust and respond to changes in customer preferences, market conditions, competitive actions and the status of operations? These questions will direct the investment and scope of Big Data projects in 2015.

.

Prediction No. 3: Self-service Big Data will go mainstream

In 2015, IT will embrace self-service Big Data to allow business users to access Big Data. Self-service empowers developers, data scientists and data analysts to conduct data exploration directly. Previously, IT would be required to establish centralized data structures; this is a time-consuming and expensive step. Hadoop has made the enterprise comfortable with structure-on-read for some use cases.

In 2015, advanced organizations will move to data bindings on execution and away from a central structure to fulfill ongoing requirements. This self-service speeds organizations in their ability to leverage new data sources and respond to opportunities and threats.

.

Prediction No. 4: Hadoop vendor consolidation: New business models will evolve while others will exit the market

We are now 20 years into open-source software (OSS) adoption, which has provided tremendous value to the market. Technologies mature in phases. The technology lifecycle begins with innovation and the creation of highly differentiated products. The lifecycle ends when products are eventually commoditized. Edgar Frank “Ted” Codd created the relational database concept in 1969, with that innovation leading to Oracle’s IPO in 1986 and commoditization beginning with the first MySQL release in 1995. So, historically, database platform technology maturity took 26 years of innovation prior to seeing any commoditization.

Hadoop is early in the technology maturity life cycle—with only 10 years passing since the seminal MapReduce framework white paper was published by Google in 2004. Hadoop adoption globally and at scale is far beyond any other data platform just 10 years after initial concept. Hadoop is in the innovation phase so vendors mistakenly adopting “Red Hat for Hadoop” strategies are already exiting the market, most notably Intel Corp. and soon EMC Pivotal.

In 2015, we will see the continued evolution of a new, more nuanced model of OSS to combine deep innovation with community development. The open-source community is paramount for establishing standards and consensus. Competition is the accelerant transforming Hadoop from what started as a batch analytics processor to a full-featured data platform.

.

Prediction No. 5: Enterprise architects will separate the big hype from Big Data

As organizations move quickly beyond experimentation to serious adoption in the data center, enterprise architects will move front and center into the Big Data adoption path. IT leaders will be vital in determining the underlying architectures required to meet service-level agreements (SLAs), deliver high availability, business continuity and meet mission-critical needs. In 2014, the booming ecosystem around Hadoop was celebrated with a proliferation of applications, tools and components. In 2015, the market will concentrate on the differences across platforms and the architecture required to integrate Hadoop into the data center and deliver business results.

.

2015 Technology Predictions graphic courtesy of SiliconANGLE