Enterprise developers and data scientists often build small proofs-of-concept for Big Data analysis projects, involving relatively small subsets of enterprise data, on public cloud platforms. But when those pan out, the development team builds the production-level system in-house. That makes sense when most or all of the data is generated in-house, but it may be adding unnecessary cost when large amounts of third-party data that originates in the Cloud is merged with corporate data, writes Jeff Kelly in his latest Wikibon Peer Incite.

In an analysis that comes out of the April 23 Peer Incite Meeting on the impact of cloud computing in general and AWS specifically on internal IT organizations (ITOs), Kelly writes that developers should not automatically keep their Big Data analysis in house. During the meeting, Jason Mendenhall, EVP for Cloud at Switch Communications, said that keeping Big Data analysis on a cloud platform has two major financial advantages. First, a major IaaS provider such as Switch’s SuperNap colocation data center can provide the heavy-duty compute resource required to analyze petabytes of data. Second, when much of that data starts in the cloud, it is easier and cheaper to bring the analysis to the data rather than trying to move the data to the analysis. Moving 2 petabytes of data at Internet speeds takes about 10 days under optimal conditions.

However, Kelly writes, the decision on where to build the system should involve more than purely financial issues. Security, privacy, network constraints, data control and location, and QoS issues involving cloud providers should also enter into the decision. The decision, therefore, should not be automatic either way but should be based on a thorough cost/benefit analysis that takes all the relevant considerations into account. But, he says, data transfer and network issues definitely should not be forgotten when making that decision.

The full Alert, like all Wikibon research, is available free of charge on the Wikibon Web site. IT professionals are invited to register for free membership in the Wikibon community. This allows them to comment on research and post their own tips, questions, Professional Alerts, and white papers. Membership also comes with invitations to the periodic Wikibon Peer Incite meetings, at which your peers discuss how they use advanced technologies to solve business and IT technical problems.