How do you use declarative provisioning to manage DevOps infrastructure?

Quick answer: declarative provisioning means describing the infrastructure you want (a config file stating "3 web servers, this database, this network") and letting a tool like Terraform or CloudFormation work out how to get there, rather than writing imperative step-by-step scripts. The benefit is that the same file is both documentation and the single source of truth, and re-running it is safe because the tool only changes what's actually different from the target state.

Hi, welcome back to the channel — today we're talking about infrastructure as code: what it is, the benefits, the tools that help you handle it, and managing it day to day.

What is infrastructure as code?

To me, infrastructure as code is storing your configuration as code that can be tested, deployed, and maintained in a repeatable way. We want infrastructure we can repeatedly install and deploy as many times as we need — tear it down, set it up again. Some people tear down their whole dev environment on a Friday, because there's no point paying for it over the weekend if you're on a cloud provider — that's one way cloud can be cheaper than running your own data centre — then Monday morning, an automated process creates it all from scratch again.

We can do this because the infrastructure is defined in code, which means we can test it — statically, or by running real tests against it — and deploy it again and again. Why does this matter? If it's repeatable, it's less effort. Are you afraid your server might fail or get deleted? Not if everything's backed up — keep the data safe, keep the configuration safe, and you can restore everything exactly as it was.

What counts as infrastructure?

Servers, networks, the rules governing access and security around them, where the logins go. Application configuration — what your application needs to run, the name of the environment, the subnet, even the subnet definition. Servers where the code runs — we can tell the code it's scalable, what the minimum number of instances is (one, two, three) and the maximum we want to spend on scaling up. When we scale down, we lose any data on those servers, so where do we back that up to? Most of the time it's just the logs — we're not editing code live on production servers, so we keep those logs and the monitoring data somewhere safe, and the server itself can be destroyed easily. Networking matters too — servers need to survive in a network, they're pretty useless if they can't talk to anybody else.

What does the code look like?

Some tools come with their own language, like HashiCorp's, and others just use YAML, depending on the tool. Let's talk about the tools: Terraform, Ansible, and something to run the pipelines, like a Jenkins server.

Terraform

Terraform uses HashiCorp Configuration Language, and can go cross-cloud with multiple providers and modules. Providers expand core Terraform with functions for a specific upstream API — AWS, Google Cloud, Azure, all have APIs we can manipulate through a program, and in this case that program is the Terraform provider. The AWS Terraform provider talks to the AWS API, so using Terraform, talking to that provider, we can create and destroy AWS resources; similarly, a Google Terraform provider talks to the Google API. There are many, many providers available, from hardware to software — there's even a JFrog Artifactory provider, so you can configure your local Artifactory instance through Terraform code.

So we've got Terraform and a provider that knows how to talk to the API — but what tells it what to actually do? That's defined in modules. You don't have to use modules — you could have one massive file with a main file, outputs, and variables all together — but that gets unmanageable fast. Most configurations I see have a main.tf with outputs and variables, defining which modules to pull in, and each module in turn has its own main file, outputs, and variables, so you can tell each module what you want, it does its thing in the main file, and reports back what it did via its outputs. Terraform allows a lot of flexibility in how you split files — in fact, when it runs, it concatenates all the .tf files together and runs them as one — but for us humans, it's better to name things sensibly, splitting a module's infrastructure component from its security or access component, say.

Ansible

The next tool that often comes up alongside infrastructure as code is Ansible. Ansible does something slightly different to Terraform — it's configuration management. It also uses YAML, and has a similar concept to modules, except in Ansible they're called roles. A role structure has its own convention, again best practice to separate things out — at bare minimum it just needs a main.yaml, which has to exist, but you can expand it with separate YAML files for more detail. For example, the main YAML might detect whether the operating system is CentOS or Debian, and include the relevant file accordingly — most roles are set out this way to be cross-compatible with multiple operating systems.

Each role has a given goal — managing a Jenkins deployment, or a web server, say — and you can combine multiple roles into one host configuration, so the host says "okay, I'm a Jenkins server, and I need whatever else that role requires" (Jenkins has dependencies, so you might also have a role that says "I need to be a server capable of running Java").

Managing it as code

Because it's code, we can use our normal code tools to manage it — git, and because it's in git, we can execute it in pipelines through our CI/CD server (ironically, you could set up a Jenkins server using another Jenkins server — probably not the best idea, a chicken-and-egg problem, but I digress). Once we've got the code, we can version control it, manage it, share it, and run it as we see fit — and if it's in a CI/CD pipeline, we can test it too, using the testing tools available in both Terraform and Ansible.

In summary

Terraform, HashiCorp's tool, runs infrastructure as code that we store in a git repository and run through a pipeline system like GitHub Actions, GitLab, or Jenkins. This creates a remote state file, which we need to keep safe, so Terraform can tell what's changed and what hasn't. Through its providers, Terraform talks to whatever API it needs — GitHub, AWS, Google Cloud — and we can run this code over and over to keep everything up to date, or scale down by running init, plan, and apply, and then destroy at the end of the week, ready to build it all again Monday.

Ansible then comes along to help configure the servers Terraform created — we add hosts to an inventory (a list of hosts and the roles they have), put those together in a playbook, and the Ansible server uses the playbook to assign roles to servers in the inventory. Once Terraform has created the servers, Ansible configures them so they're ready for work.

These aren't the only tools that do these jobs — if you don't need to go cross-cloud, you might define things directly in CloudFormation for AWS, or the Azure/Google Cloud equivalents. For configuration management, besides Ansible there's Chef and Puppet, which are closely related, or SaltStack if you want to go a bit left-field.

I hope this has been useful — any questions, put them in the comments below and I'll answer as best I can. Until next time, may all your deployments be smooth.