Infrastructure automation is no longer optional.

It is at the core of many of the major changes happening in enterprise information technology. DevOps requires it. Modern infrastructure technologies such as containers and public cloud infrastructure-as-a-service enable it. Managing the coming proliferation of huge numbers of connected devices will not be possible without it.

So it’s no surprise that IT organizations are rushing to automate their infrastructures. But they often struggle to automate effectively. Many teams are starting from zero, transitioning from processes that are entirely manual. And for these organizations, automation can be a surprisingly heavy lift.

Infrastructure automation doesn’t just involve technology. It also brings real changes for people, policies and processes. Your automation strategy needs to solve both technical and nontechnical problems. Here are six steps you can take today toward an automated future:

Build a CIA pipeline

State-of-the-art infrastructure automation employs a delivery pipeline designed to implement a coordinated sequence of tasks, while also testing each outcome. We call this “continuous infrastructure automation,” or CIA. It extends the software development practices of continuous integration and continuous delivery or CI/CD to infrastructure.

In the CIA pipeline, each stage includes an action step that performs a task, plus a validation step that tests whether the action has occurred as expected. If the test passes, then the pipeline proceeds to the next stage. If it fails, then the pipeline halts. Thus, each change is tested for production readiness before it’s deployed. With a CIA pipeline, you can deploy incremental changes rapidly but safely.

Choose your tools

There is no one tool that will build and execute the entire CIA pipeline end-to-end. Gartner clients use an average of eight tools throughout the pipeline. There are a variety of popular open-source choices.

Public cloud providers now offer their own native automation tools, such as Amazon Web Services Inc.’s WS CloudFormation, Microsoft Corp.’s Azure Resource Manager, or Google LLC’s Cloud Deployment Manager. And your incumbent data center infrastructure vendors probably sell their own automation tools, too. Any or all of these may end up in your toolbox.

Modernize the infrastructure

Modern infrastructure is software-defined. It makes resources available programmatically, but more than that, it defines abstractions to which software can refer. You don’t have to manage all the low-level details; whatever you need is a simple application programming interface or API call away. Software-defined infrastructure means there’s less work your automation pipeline needs to do, and fewer interfaces you’ll need to wire up to your automation tools.

One way to get the benefits of modern infrastructure is to run in the public cloud. But that is not the only way. Innovations such as hyperconvergence and software-defined networking, among others, aim to bring cloudlike, software-defined infrastructure to the data center and the edge.

Automate security and compliance too

Any state-of-the-art automation pipeline incorporates security and compliance controls. You can build software composition analysis and vulnerability scans into your automated test suites — and you should. Meanwhile, a new generation of policy-based management tools is emerging.

With these new tools, you can describe your corporate policies programmatically, and your automation tools will enforce them. This is an emerging area, but tools such as Chef InSpec, HashiCorp Sentinel and the Open Policy Agent are a few examples that do this today.

Start small, and grow from there

For a traditional enterprise IT organization, infrastructure automation may be a multi-year effort. You can’t get there in one leap. Rather, you’ll need to take a page out of the lean product development playbook: Build your minimum viable product, then expand its capabilities over time.

Most infrastructure and automation teams begin by automating server provisioning. This is a sensible place to start, since fully automated server builds are a prerequisite for many other data center automation tasks.

Develop new skills, roles and responsibilities

Many veteran sysadmins lack automation expertise. Gartner clients often tell us this skills gap is their biggest obstacle to automating infrastructure. You don’t have to become a software developer. But you will need to understand the basics of scripting, object-oriented languages, version control, code review, and the development lifecycle.

The good news is, automation expertise is among the most desirable (and lucrative!) skills for technical professionals. Moreover, automation is emerging as its own specialty within IT, spawning new kinds of jobs for dedicated automation experts. Appointing an automation architect is a great place for your team to start. In fact, Gartner predicts that by 2022, 60% of large global enterprises will have an automation architect, up from less than 20% today.

There is no shortage of hype around infrastructure automation, and some persistent misconceptions:

• You can’t buy automation. No one tool will automate your entire data center. More importantly, no tool will devise your automation strategy for you, or deal with the policy and organizational implications of an automation project.
• You probably won’t save any money. Infrastructure automation tends to shift labor, not reduce it. It rarely lets you cut headcount. (Good news for engineers, but maybe bad news for bosses.) Automation makes infrastructure better, not cheaper.
• Some things shouldn’t be automated. Don’t try to automate your way around bad architecture or inefficient processes. Re-architect before you automate. You’ll get better results.

But as is usually the case, beneath all the hype, there is truth. Even if automated infrastructure is not your present, it will be your future.

For more information about automating your infrastructure, Gartner is offering a complimentary Webinar with me at 11 a.m. EDT Oct. 29: The Field Guide to Infrastructure Automation.

Paul Delory is a senior research director on the Data Center and Cloud Operations team within Gartner Technology Professionals Research. He provides deep technical expertise to systems administrators and the vendors that serve them. His research centers on automation and next-generation infrastructure.

Photo: Equinix