The networking space is in the midst of significant changes. The starting point is the transition to higher speeds, with most customers finally moving to 10Gb Ethernet (almost a decade after the standard was ratified), and 40Gb and 100Gb solutions starting to become available. But the real drivers that make networking strategic to IT are the trends of virtualization, convergence and cloud. As has been the case for the last decade, the conversation of the marketplace starts with Cisco. Cisco has been talking about “Data Center 3.0” for over three years; today’s Cisco announcement (here’s the PR) is delivering on a number of pieces of the vision. As Cisco pushes into adjacent server market, it finds the networking marketplace more competitive than it has been in many years.

Virtualization is Driving Networking Change

Server virtualization is a game changer for much of IT. What started out as increasing the efficiency of servers quickly moved into transforming resources into a pool of resources that had mobility and this had a ripple effect on storage and network architectures. Dense, multi-core server environments with high utilization have put pressure on existing architectures. On the storage side, this has led to the use of SSD drives and increased the utilization of unified storage, plus it was a significant driver for the creation of Fibre Channel over Ethernet (FCoE). On the network side, virtualization, specifically applications like VMware vMotion, requires more bandwidth between servers (see Greg Ferro’s excellent explanation of “East West” Bandwidth), which is leading vendors to flatten the traditional 3-tier architecture and look for alternatives to Spanning Tree Protocol. Cisco is at the leading edge of supporting virtualization; both the Nexus (switch) and UCS (server) product lines were designed to intersect where this trend is going. The graphic below gives a good visualization of how Cisco believes that it is maintaining a mix of sustaining and disruptive innovation.

Your Next Switch

Convergence, virtualization and cloud are all great to talk about, and hopefully customers are taking the trends into consideration, but at the end of the day, it comes down to what 10Gb Ethernet infrastructure to standardize on. While Cisco’s Catalyst product line does support 10GbE, the Nexus family is the flagship for new features and functionality. Today’s announcement includes many enhancements for the Nexus line:

• An extensive expansion to FCoE support (more details in the section below) including on the Nexus 7000 and MDS lines
• New Nexus 3000 for the HPC market
• New Nexus 5548 and 5596 switches that include Unified Ports

Cisco looks not only to maintain its dominance in networking, but also to extend the reach and usage of Ethernet. High Performance Computing (HPC) market is a niche that is driven by latency and cost. Historically dominated by proprietary standards, the HPC market is now primarily split between InfiniBand and Ethernet. As Ethernet vendors continue to drives down latency and cost, it will limit the market that InfiniBand can address. Arista Networks is known for being successful in the HPC market and now Juniper QFX3500 (with claims of less than 5μsec latency) and Cisco with the Nexus 3000 will position Ethernet products against Mellanox and QLogic InfiniBand solutions.

Big Steps for Converged Networks

While most enterprise customers that I speak to in the Wikibon community agree that a single network is desirable, most administrators will push against changing from the separate SAN and LAN architectures until they have to. From the vendor perspective, the next-generation, dense, blade servers that are optimized for virtualization could not be created without a single network. FCoE was an enabling technology for the UCS architecture and is also embedded in new generations of HP and IBM blade servers. When FCoE is built into components, there will be pressure from senior management to use it rather than buying separate infrastructure. It is not that FCoE will be free (see my post on Intel’s Open FCoE for more on this), but the cost of adding a license or line card should be much less than separate infrastructure. With the new announcements, it is without a doubt that Cisco has the broadest converged networking portfolio, including complete coverage with FCF (FCoE Forwarder, basically FC switch functionality equivalent for FCoE) in embedded (UCS), edge (Nexus 5000), core (Nexus 7000) and FC switch (MDS) product lines. The final piece in the embedded story will be later this year when Intel Sandy Bridge servers hit the market as they will have much broader support for LOM solutions that support FCoE. The really big step for FCoE configurations is that multi-hop capability is now as broad as existing FC solutions. Cisco’s solution is all standards based (FC-BB-5 ratified in June 2009) VE_port to VE_port communication (see Erik Smith of EMC’s E-Lab write-up of VE_port). I worked at EMC through the development of FC multi-hop solutions which took years to move from single switch, two switch, four switch mesh and ultimately iterations of multi-hop; it is gratifying and great for customers to see this evolution accelerated. While the goal of convergence is to move to a single network, FCoE allows FC and Ethernet solutions to be delivered and maintained as a single solution, and especially with technologies like Unified Ports, customers have the freedom to mix and match as needed. Storage people may not be clamoring for FCoE and network teams might not want the responsibility of managing a converged solution, but this is not a trend that can be avoided, so customers need to be engaged.

Source: Stuart Miniman, Wikibon 2011

Changing Roles and Management

Cisco is focused on helping IT organizations break down silos with policy management. The convergence of compute and network (UCS) and of storage and network (FCoE) both require new thinking and new tools. Better management tools are at the top of customer requirements. Cisco has updated its Data Center Network Manager (DCNM) tool to allow for a single pane of glass for LAN and SAN. DCNM can start at a virtual machine (using VMware APIs) and manage from the compute through to the storage, allowing for enhanced troubleshooting and provisioning. While customers can continue to segment management by role and must make sure that storage administrators can keep data safe (best practices are drastically different between LAN and SAN), orchestration tools are important for more efficient IT environments. On a similar note, Cisco announced its intent to acquire newScale, a product that it previously OEMed for cloud automation/self-service portal software. Cisco technology sits in the middle of such a broad spectrum of environments; management and deeper integration with partner frameworks will continue to be critical.

The New Data Center Regime

Cisco is at the leading edge of convergence and is looking to ride with existing customers (and the army of Cisco Certified professionals) on the wave of virtualization and cloud to continued leadership in networking and new revenue opportunities. Cisco is continuing to push innovation and differentiation to help maintain marketshare and margins against competition. The refreshed offering helps Cisco highlight areas of strength such as in storage environments. One of Cisco toughest challenge is getting customers to move fast enough towards the vision that is now available to purchase. CIOs should take the time to understand new architectures options from Cisco and others, and strongly consider deploying next-generation solutions.

[Cross-posted at Wikibon Blog]