How do you bring order to chaotic DevOps infrastructure using Ansible?

Quick answer: Ansible brings order to chaotic infrastructure by turning manual, undocumented server configuration into version-controlled playbooks that describe the desired state — so instead of remembering which of twenty servers has which manual tweak, you have one source of truth you can re-run safely, and a git history of exactly what changed and when.

Hello, welcome to the channel, I'm Neil, your DevOps expert, and I also run a group called tech-answers.club — I'll tell you more about that later. Today I'm going to talk about Ansible: what it is, what it's used for, and how you can get loads of benefit from it.

Infrastructure takes management — if it doesn't get managed, it turns into a giant mess. If you've ever worked in a data centre, you might have seen those pictures of bad cable management — the back of a rack with cables of different colours in a spaghetti mess, a bit like some code I've looked at. With proper cable management it can look nice, neat, and tidy, and make it really easy to maintain — and that's what Ansible can help us do for our servers.

What is Ansible?

Ansible is a tool for configuration management. It can be saved into git repos, so it counts as infrastructure as code. It can also manage infrastructure like AWS, but I'd recommend Terraform for that — Ansible really excels at looking at the configuration and keeping everything up to date on your servers: install, remove software, update configuration, and other housekeeping tasks.

What is it made up of? An inventory — that's your servers. Playbooks — a link between the servers and what you want them to do. Roles — a group of tasks. The tasks are what actually configure the server. Variables, because we want to keep the variables separate from the code. And groups, where we can group the hosts together.

The fundamentals are that this is infrastructure as code — everything is, in theory, written down in code, and we execute Ansible playbooks to make that a reality on our servers. One of the benefits of having it in code is we can run it over and over again, not just on other servers but even on the same server, so we can prevent configuration drift.

Playbooks and roles

A playbook is a YAML file that links the inventory to the tasks through the glue of roles — it tells us what server groups should be applied with what roles, and the roles contain the tasks. Roles can also be made up of modules — reusable pieces of code, so if we've written some tasks and put them into a role, we can store those as a module. Provided we keep the variables separate, we can share those modules with other parts of the business, colleagues, or friends — or they can share theirs with us, so we don't have to do the hard work if they've already done it. Take a look at Ansible Galaxy for more examples of this.

Variables are important because not every server is the same — some have unique attributes, like the server name, or, if they're hosting a website, the website address or the name of the SSL certificate. We can group hosts (the name for servers in Ansible-land) into groups, and apply variables at the host level and the group level — at the group level we can say "we want these hosts to have this role," and that's what the playbook does.

A quick demo

I've got a server I created, and an Ansible playbook called "docker-host" — it's got several files in it, a README of course, and the playbook itself. What we need is the hosts file — that contains variables we can apply to the groups. Let's create one: we've got our host, called "ansible-demo," and all hosts need to be in a group, so let's put it in one. Because we're going to SSH in as a host, let's make sure we can contact it over the network by pinging it — there we are, and that's what the Ansible playbook command is going to look like.

Let's SSH into the server — I used another Ansible playbook to configure this server so I could SSH onto it with my username — and try running the docker command, and it's not there. So let's look at the actual playbook — it also contains an NTP role, but we don't need that since this server already has NTP set up, so we'll comment that out, and we're not hosting anything so we'll get rid of Let's Encrypt as well. Essentially all it's going to do is install Docker for us.

So let's run the playbook — it removes old Docker versions, adds the dependencies we need and the repo, then finally installs Docker. Took about 25 seconds on my machine. Like I said, we can run these things multiple times and that's okay — it just reasserts the configuration you want, and everything else is skipped if it doesn't need to do any work, all driven off the host file we created earlier. The playbook contains roles, the roles contain tasks, and the tasks start off with a main file, where we can do stuff like check the operating system, so we can have different tasks for Debian and different tasks for Red Hat because they use different commands, and finally the tasks at the bottom level do what I mentioned — installing Docker and, most importantly, starting the Docker service. Head back over to the server, and lo and behold, Docker now exists.

Scaling it up

That's a very simple example, but what else can happen in the real world? Doing this on one server is fair enough — you could probably achieve the same by SSHing in directly, not much time saved — but if you've got multiple servers, say 20 web servers, you wouldn't want to log into all 20, set up accounts, add packages, and create configuration files by hand, that would take a long time, especially in a dynamic environment. If you want to reconfigure all those servers, having this playbook just to play, it's all done in about 35 seconds.

You can also use the playbook across multiple environments — many of my customers have a dev environment, a testing environment, a staging environment, and a production environment, and you can use the same code against all of them, ensuring consistency across all of those, which is very important, and makes things easier.

Pitfalls

But there are some pitfalls. Ansible does allow you to write custom code — run a bash script or something — however, if done badly, or without awareness, it means you lose the ability to run it idempotently, over and over again. If you're just running a bit of bash code that appends an extra line to the bottom of a file, every time you run it, it sticks another line on, and before long you've got six, seven, eight identical lines, which is probably not what you want. With mindfulness you can make sure it handles repeated runs — the inbuilt modules already have this in mind.

Dynamic inventories — I've mentioned this already. Let's say our servers are in AWS — we don't necessarily know the hostname or even the IP address in advance, so we can run a utility script that goes off into our AWS account and gathers what the servers are. We can put tags on those servers, and those tags form group names inside the Ansible inventory, so we can target the groups in the same way we always do, based on the server tags.

Secrets — what's a secret? We've got our variables, and we don't want variables containing sensitive information inside our repository. Ansible Vault enables us to encrypt those secrets so we can store them in the repository and they're still relatively safe. This way we can use continuous deployment — running the same code over and over with our secrets, reasserting what the server should look like, keeping database usernames, passwords, and other important information away from prying eyes.

Performance

Ansible works by SSHing onto the server, doing the thing, and exiting again, so there are a few things we can do to speed this up:

• Forks — enables us to contact more than one server at a time. If we're configuring those 20 servers, we can run with forks: 20, hitting all 20 servers at the same time.
• Async operations — other configuration management tools like Puppet run async by default, and you have to massage them to do things in the right order. Ansible does things in order by default, but you can tell it to run certain tasks asynchronously, so it doesn't have to do them one after the other. This can speed up your playbooks.
• Disable host key checking — if you're in a trusted environment, this can save you one or two seconds on every SSH connection, and given Ansible connects, does the thing, and disconnects for every task, that can add up significantly across a playbook.

So that's Ansible — we talked about the inventory, the playbooks that tie the inventory to the roles, the roles and Ansible Galaxy for modules, variables that let us personalize servers (which we can even encrypt with Vault), and groups that keep our hosts together and tidy.

If you've got any questions, put them in the comments below, and come and ask me at our weekly Q&A every Wednesday at lunchtime — free, at tech-answers.club. Hope that's helped you understand what Ansible is, and until then, may all your deployments be smooth.