Despite its strong association with the cloud, most Big Data deployments are in private data centers, writes Wikibon Principal Research Contributor Jeff Kelly in “For Big Data to Thrive in the Cloud, More Managed Services Required”.

This creates a couple of issues. First, many Big Data use cases are built on large volumes of data that reside in the Cloud, not the corporate data center, and moving that data is difficult and time consuming. Second, building out an infrastructure to support Big Data analysis beyond the proof-of-concept level is expensive, requiring large amounts of hardware that must be crammed into often already crowded data centers. Unless the enterprise has a near constant need for Big Data analysis, much of that infrastructure will lie idle much of the time, generating little value while it depreciates.

One reason that more of these initial Big Data trials are not done in the Cloud, Kelly writes, is a lack of Hadoop-based, bare-metal platforms on which to build them. He identifies three:

1. Google’s Big Query,
2. The AWS Big Data engine, and,
3. Treasure Data (see embedded video below).

While these are good choices, more are needed to fill out the market. He suggests that vendors should build analysis platforms on top of multiple IaaS environments, such as IBM SoftLayer, which specifically includes a bare metal environment.

Splice Machine

x

One new vendor looking to enter this market is Splice Machine, the latest venture of Monte Zweben, whose projects have included Red Pepper Software and Blue Martini. Zweben says Splice Machine, which is currently in private beta, is the only NewSQL database engine built on Hadoop and combining the best aspects of both. Designed for near real-time transactional environments, he describes Splice Machine as a very high volume solution and says that one of the early users moved a very large database onto it when Oracle “hit the wall”. He says Splice Machine, which will enter the public market in the next few weeks, is already running on AWS, and he is exploring other cloud platforms.

Kelly recommends that Big Data practitioners explore cloud services to free themselves from the challenges of provisioning hardware, tuning systems and maintaining performance or alternatively building their own Big Data analysis platforms on IBM/Softlayer or other bare-metal services.

Kelly’s full analysis, like all Wikibon research, is available without charge on the Wikibon Web Site. IT professionals are invited to apply for free membership in the Wikibon community, which allows them to post comments and questions on research, publish their own tips and analysis, and participate in Wikibon research.

photo credit: PhOtOnQuAnTiQuE via photopin cc