An unprecedented group of technological and market forces are forcing a radical restructuring of the IT industry at a blinding pace never seen before in the history of IT, and indeed not in human history since the end of WW II.

The closest thing to it in recent history was the PC revolution of the 1980s. In 1980 Digital Equipment Corp., Data General, Wang, and Prime Computer were among the fastest growing companies in the industry. By 1990 only DEC was left, and it finally disappeared in 1998, when what was left of it was bought by Compaq, which itself was then bought by HP. In 1980 pundits were predicting that AT&T and IBM would dominate the computer/communications era. By 1990 both were heading toward major restructuring of their businesses. And the entire IT infrastructure was shifting radically as the network usurped the place of the mainframe at the center, and everybody scrambled to move to client/server.

That was the impact of two new technologies, the wired Ethernet and desktop (and later laptop) computer. This transformation is being driven by the convergence of several major new technologies, each of which by itself is as transformative as desktop computing in the 1980s. The drivers of radical change include Moore’s Law, Open Source, converged hardware, software-defined infrastructure, hyperscale architectures, public cloud services, and the new mobile end-user lifestyle and expectations that these technologies support.

Exactly how radical this transformation will be, and what vendors will be left standing in a decade, is uncertain. One thing is sure, the IT industry in 2023 will be very different from the industry today, and every vendor, from the newest startups to the giants of the industry, will spend the decade scrambling not just for increased market share but often for survival.

Hardware Revolution

All this may sound fantastic, so let’s take a look at the implications of each of these drivers. Moore’s Law –that IT computing power doubles every 12-18 months — is not a new idea. But the power of this arithmetic progression is only being felt now. In 1980 a doubling of computer power every year was not that big a deal. Today it is a prodigious leap. The result, as IBM Fellow and Chief Architect for PureApplication Jason McGee told me, is that modern servers have compute power to burn. The gating factor today is network speed.

So what are the vendors doing with all this extra compute power that is trapped inside the machine? They are automating low-level management tasks that until recently were done manually by IT techs. The immediate result: companies need fewer of those expensive bodies. Converged hardware amplifies this by creating full “datacenters in a box” that arrive fully integrated with unified, highly automated management built in.

This trend is being magnified by software-led infrastructure. We are fast approaching the point where entire virtualized data centers will be common. As software-defined networking pioneer Martin Casado explained a year ago in TheCube from VMworld 2012, the immediate effect of this is to unify management of all physical devices and move that management to the edge of the network. The ultimate result will be that entire fully virtualized datacenters will be routinely managed from a single unified “pane of glass” by a single person, and events that a decade ago would have been full emergencies will be handled automatically with little or no human involvement. And the operations staff for the data center will be reduced to four or five individuals total. Tomorrow’s data center floor will be as devoid of humans as today’s automated automotive manufacturing floor is.

The next logical step is hyperscale. This architecture,  created by the Internet giants including Google, Yahoo!, Facebook, and Twitter, uses plain vanilla whitebox hardware from Taiwan that is completely virtualized and managed by the highly automated software layer. The boxes are so cheap that hyperscale companies have replaced break/fix with break/throwaway. The hardware isn’t worth repairing. The question here is will hyperscale remain exclusively a creature of the big Internet vendors, or will the big internationals start adopting it. Wikibon CTO David Flouer says it is only a matter of time before the latter happens. I would add that we will start seeing it sooner that most people expect, possibly as soon as 2014.

Meanwhile the SMB market is rapidly moving to the public cloud, from public cloud. Ninety percent of small companies — basically any company that does not gain competitive advantage from specific IT systems — have no reason to maintain any in-house IT at all. They are better served by SaaS vendors that can deliver better service at a lower price while eliminating the distraction of managing IT. And at some point those services themselves reach a size where they adopt hyperscale.

But even at the high end the logic of public cloud is having an impact. A few years ago IT guys told me they would never use public services. Now they are all embracing hybrid cloud, and most are using at least some SaaS services and/or moving some core systems to IaaS providers, including those run by traditional hardware vendors such as IBM and HP.

Software Threats

Meanwhile, core software markets are threatened by Open Source. Linux has already taken a big bite from the OS market. In the early 1990s Microsoft was doing a boom business selling Windows Server for low-level servers, particularly for serving pages on the Internet and similar workloads. Then Linux appeared. Microsoft, of course, still is a player in the datacenter, and Wikibon’s Scott Lowe says it dreams of taking over the full datacenter. Good luck with that. Actually it is caught in a squeeze as Linux eats up the datacenter from the bottom while  vendors like IBM and HP hold the high end, particularly in large enterprises. IBM sells Linux on mainframes. And guess what OS the hyperscale servers use. Not Windows Server.

The next software market to watch is databases. Hadoop is the big threat here. Today the common Big Data strategy is to use the legacy data warehouse for structured data and Hadoop for all the new Big Data. But this two-database strategy has obvious inefficiencies. Hadoop can, in fact, store structured data for analysis, and it is much cheaper than those Oracle licenses. I am definitely not saying that Oracle, DB2, Informix, etc., are going to disappear. Hadoop has limited functionality and is not a good candidate for tier 1 transactional databases in particular. And Hadoop skills are still thin on the ground. But the data warehouse is at risk, and that is a chunk of Oracle’s market in particular.

The impact of mobile computing is more complex. On the one side, it promises to bring a lot of data and applications that moved to the desktop in the 1980s back into the datacenter as users ask for the same functionality on their smartphones and tablets that they have on their laptops. On the other, the increasingly mobile, and in many cases physically scattered, workforce, is dictating that increasing amounts of these resources reside in the Cloud. Here at SiliconAngle, for instance, I work with a team that includes a writer in Bangkok, another in a Manila suburb, another in Israel, as well as others in California, Texas, Denver, and Boston. I live in the Blue Ridge west of Washington, DC. Almost all our core systems live on the Internet.

So those are the forces at work. In the next installment I will take a high level look at how the industry and some of the important individual vendors are responding. Rest assured, none of what I have written here is news to any of them.