“When we moved to the new stadium, we wanted to do more with technology to create the optimal fan experience,” says Bill Haggard, director of enterprise infrastructure for the Dallas Cowboys. In the old stadium, he said in an interview on SiliconAngle.tv (http://siliconangle.tv/video/bill-haggard-dallas-cowboys-talks-virtualization-cube) with Wikibon.org co-founder David Vellante and SiliconAngle.com founder John Furrier, “We had the attitude of 70% break/fix and 30% innovation.” Im the new stadium, things were going to be different.

“For instance, we have 212 concession stands in the new Texas auditorium. Each concession stand is treated like its own restaurant. So each has to have its own Windows 2003 server on the back end.” That plus some ancillary servers would bring the total up to almost 300 physical servers just for the concession stands, plus servers for the 10 pro shops in the stadium plus 35 more in the community, and other functions.

That was an impossible environment. So instead Haggard and the Cowboys virtualized their environment. That reduced the physical environment from 10 racks down to 16 blades in two HP C7000 chassis. Not only did this cut capex expenses for all those servers and reduce the amount of cooling, power, cabling costs, and other opex, it also put the entire physical infrastructure into a much smaller footprint, saving floorspace in the stadium data center.

And the stadium is only part of the story, because the IT department also supports the 30-odd other businesses owned by the Jones family. The environment includes three data centers – one in the stadium, a second in the central merchandise facility and the third at Valley Ranch, the Jones family business headquarters. Virtually all the front-end functionality in this environment is virtualized – the only exceptions are a few special applications whose vendors do not support virtualization.

Flexibility and Disaster Recovery

This provides huge improvements in functionality and flexibility. For instance, all pricing in the concession stands and pro shops is virtual, controlled by the computer system. “If the Cowboys win, for instance, fans are more likely to buy in the pro shops,” he says. “So we can control and adjust the prices while the event goes on.”

On the other hand, if something happens to the data center, the retail outlets all can run independently. This is an important part of the Cowboys’ DR plan, since the concessions and pro shops are the closest they have to RPO Zero operations.

To protect everything else, including advanced ticket sales and the other Jones businesses, “we have a DR strategy that backs up data from each of the data centers to the others, so that if we have a problem with a mission-critical system or a disaster at any one location we can shift those operations to the other two.”

The one area that is not yet virtualized is the back-end business intelligence. This is not uncommon, and many IT shops purposely exclude BI out of concern for performance impacts on the transactional systems. Mr. Haggard, however, says the main reason they didn’t virtualize their back end was an accident of timing.

“Truthfully when we implemented all the new technology in the new stadium, vSphere was not out yet. We had to get everything up and running and didn’t have the luxury of waiting.” Now that vSphere has been proven in the market, he plans to virtualize the Cowboy’s BI in the 2011 off-season. “We will probably virtualize a subset of our back-end in a test environment to make sure that we won’t have problems and can maintain the performance we need to support our front-end operations first.”

The Secrets to Success: Careful Planning and a Good Team

Looking back on the two year virtualization experience, Mr. Haggard cannot think of much he would have done differently. However, he warns against rushing into virtualization.

“We went through about a 14-month-long process to plan out what we wanted to do, get the design right, and then start the implementation phase,” he said. An important part of that is understanding the direction of the vendors. “We needed to know their technology vision not just for tomorrow or a year ahead but seven years, ten years down the road. We needed to be assured that in three years we wouldn’t have to do forklift upgrades on everything.”

The team is vital. “You have to make sure you are getting a top-notch group of people to work with who can take your vision and understand what you need to do. You can run into issues with some consulting companies or even your own staff when someone has an ulterior motive and disagrees with the direction the organization needs to take.”

It requires effort, but the reward is worth the investment. “You absolutely do need to look at virtualization. IT can save you countless amounts of money and give you the flexibility you need for the future. But it’s something you have to plan out and do right.”