Data center virtualization creates compelling financial advantages mainly by increasing average server utilization from an average of 15% in non-virtualized environments to 85%+. However, it also has created increasingly complex environments as virtualization grows in data centers in which the addition of a new VM or changes to underlying infrastructure can create unintended issues throughout in the environment.

In the unvirtualized environment, where each application ran on its own server and storage and only the network was shared, if an application had a service level problem, IT could analyze it in isolation, determine exactly where the problem lay and take appropriate action with little concern for the rest of the data center. In some cases, for instance if the answer was to upgrade network components, those other applications might actually benefit.

Post-virtualization, however, with VMs moving from server to server dynamically depending on demand, storage similarly virtualized, and the underlying physical infrastructure contracting onto fewer, larger boxes that are more efficient, infrastructure changes are guaranteed to have effects across the environment. For instance, today solid-state storage is often seen as the cure for all performance problems. But users often find that flash drives only move the choke point to a different place in the environment, and the investment may only lead to more expensive upgrades of other hardware. Diagnosing problems becomes very complex as a change in a VM can impact other VMs. In the highly dynamic environment, which can change hour-by-hour in response to load changes, the question may change faster than traditional analysis can find an answer.

On top of that, IT is chronically understaffed as a result of years of very tight budgets, and with the continuing instability in the global economy, driven in large part today by the situation in the European Union, companies are loath to hire new staff. As a result few IT shops have the personnel to do traditional proof-of-concept testing.

New physics

.

A new company, CloudPhysics,  created by a team from VMware and Google, are applying the techniques of big data analysis to providing a better answer for managing the complexities of virtualized environments. Two days after coming out of stealth mode and as a preview to next week’s VMworld 2012, company co-founder and CEO John Blumenthal, along with representatives of two beta testers and other experts, discussed its so far unique approach in the August 21 Wikibon Peer Incite.

Basically CloudPhysics is building a large database of anonymous performance data on the virtual computing environments of a large and growing group of organizations in multiple vertical industries. Client companies can then have various analyses run against that data ranging from benchmarking their environments against similar ones in the database to seeing what impact specific upgrades or other infrastructure changes have had in actual operating environments, to finding how other companies fixed specific problems.

One result of this, the participants said, is to eliminate the need for a great deal of time-consuming proof-of-concept analysis. It also eliminates the guesswork and wrong guesses that IT has been forced to make while attempting to manage the largely unknown virtualized environment. It allows organizations to anticipate issues and plan answers. For instance, a client can search the database to see how a move to solid-state storage impacted organizations with similar virtual environments and anticipate unintended consequences such as overloading network switches. Clients can use existing analysis of the data or submit their own questions to CloudPhysics to add to the growing library of analysis.

They also of course can contribute their own performance data to the database. This data is anonymized automatically at the source, and the data owner receives sole possession of a key that provides full control over their identification. This allows a contributor to selectively reveal its identity with its data to individual other users. The team at CloudPhysics does not know the identity of the owner of any dataset and therefore cannot accidentally reveal it.

CloudPhysics is also making access to its database available to other entrepreneurs with their own ideas of how that data can be used in different ways through the open source community.

CloudPhysics will be attending VMworld and displaying at the Fusion-IO booth.

Wikibon Peer Incite meetings focus on IT issues in the areas covered by Wikibon.org. They are public and held on Tuesdays at 12:00 p.m. ET twice a month and run for one hour. Anyone registering on the Wikibon site with a valid e-mail will receive notices with dial-in information, and meetings are also webcast live on SiliconAngle.tv. Call-in attendees have the opportunity to ask questions and add comments live.

SiliconAngle.tv will be webcasting live from the Cube at VMworld August 29-September 1, 2012.