Automate All the Things With Ansible: Part Two

Overview

This is part two of a two-part tutorial on Ansible. Part one is here. In this part, you will learn about roles (Ansible's building blocks), variables, loops, how to use roles in playbooks, and how to organize roles into a directory structure.

Roles

When you manage tens, hundreds or more servers, probably many of them need to be configured similarly. Different groups of servers like web servers or database servers will require their own special configuration, but also may share some other common functionality. It is of course possible to just copy tasks around, but this gets old really fast when dealing with a complicated infrastructure.

Ansible roles are the ticket. Playbooks can include roles. Roles can depend on other roles, and Ansible best practices recommend grouping hosts in your inventory file based on their roles. Roles are the backbone of serious Ansible-managed infrastructure. As usual, I'll start with an example and introduce many of the capabilities of roles through the example.

I like aliases and shell functions a lot because I can't remember all the arcane switches and options for each command, and also because it saves a lot of typing. I also like to have some tools like htop andtmux on every server I log in to.

Here is a file that contains some of my favorite aliases and functions. I'll call it '.gigirc'. By the way, if you ever wondered what the 'rc' suffix stands for in all those rc files, then it stands for 'Run Commands'.

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89

alias sv='sudo vim' alias py='python' alias pi='pip install' # Directory listing in a nice format alias lla='ls -lAGh' # Find zero size files alias lsz='find . -type f -size 0 -exec ls {} \;' # Remove all *.pyc files recursively alias rmpyc='find . -name "*.pyc" -print0 | xargs -0 rm' # Disk usage that also sorts the results by size and saves to a file alias dus='du -Pscmx * | sort -nr | tee disk_usage.txt' alias g='git' alias gci='git commit -a' alias gcia='git commit -a --amend' alias gb='git branch' alias gbd='git branch -D' alias gco='git checkout' alias gpu='git pull --rebase' alias gg='git grep -i' alias gs='git status' alias gd='git diff' alias gl='git log --oneline' # Show all untracked files and directories in current dir alias ng='git clean -n -d .' # Fetch and track remote branch function gfr {     git checkout --track -b $1 origin/$1 } # Create remote git branch (and local too) from master function gbr {     gco master     gb $1     g push -u origin $1 }

Let's define a role called 'common' that creates a user called 'gigi', adds a public ssh key, copies the '.gigirc' file and adds a line at the end of '~/.bashrc' that runs this file and finally installs the common packages vim, htop and tmux (defined in the 'vars/main.yml file'). 

I will introduce a lot of new stuff here: four different modules, variables, and loops. Also, roles are typically spread across multiple files in a standard directory structure. I'll show you a couple of files and then explain about the directory structure. Here is the 'tasks/main.yml' file:

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

--- - name: Create a user named gigi   user: name=gigi    - name: Add public key   authorized_key: user=gigi key="{{ lookup('file', '~/.ssh/id_rsa.pub') }}" - name: Copy the .gigirc file to the home directory of the new user gigi   copy: src=~/.gigirc dest=/home/gigi/.gigirc owner=gigi group=gigi mode=0644 - name: Run .gigirc from .bashrc   lineinfile: dest=/home/gigi/.bashrc line="source /home/gigi/.gigirc" - name: Install common packages   apt: name={{ item }} state=installed update_cache=true force=yes   with_items: COMMON_PACKAGES

And here is the vars/main.yml file that contains the definition of the 'COMMON_PACKAGES' variable used to specify which common packages to install.

1 2 3 4 5 6 7 8 9

--- COMMON_PACKAGES:   - vim   - htop   - tmux

Modules

The user module can manage user accounts. Here I use it to create the user 'gigi'.

The authorized_key module is for adding/removing SSH authorized keys. Here I use it to add my public key for the 'gigi' user.

The lineinfile module can be used to replace or add single lines to a file. In this case, I use it to source the '.gigirc file' from '.bashrc', so all the cool aliases and functions in '.gigirc' are always available in any interactive session.

Finally, the apt module has tons of options for managing apt packages. Here I just install some common packages.

Variables

The COMMON_PACKAGES you see in the last task for installing common packages is a variable. Ansible lets you use variables defined almost anywhere: playbooks, inventory, roles, dedicated files, and even environment variables. There is a lot more information about variables in the documentation.

Loops

Ansible is declarative, so it doesn't support explicit loops. But there is a plethora of with_xxx that allows you to perform repeated operations on some structure like a list of users, packages. or lines in a file. You can also repeat operations until some condition is true or get the index of the current item. Additional information can be found in the documentation. 

Role Directory Structure

Here is what a typical role directory structure may look like:

common

├── handlers

│   └── main.yml

├── meta

│   └── main.yml

├── tasks

│   └── main.yml

├── templates

└── vars

    ├── Debian.yml

    ├── Ubuntu.yml

    └── main.yml

The 'tasks/main.yml' file is the where all the tasks are defined. Each task corresponds to an Ansible command that typically uses a module.

The 'meta/main.yml' file will contain a list of other roles that the current role depends on. Those roles' tasks will be executed before the current role, so it can be sure all its prerequisites are met.

The 'handlers/main.yml' file is where you keep your handlers, like the handler you saw earlier that starts Nginx after installation.

The templates directory is where you keep Jinja2 templates of configuration and other files that you want to populate and copy to the target system.

The vars directory contains various variables and can conditionally contain different values for different operating systems (very common use case).

It's important to note that Ansible is very flexible and you can put anything almost anywhere. This is just one possible structure that makes sense to me. If you look at other people's directory structures, you may see something completely different. That's totally fine. Don't be alarmed. Ansible is not prescriptive, although it does provide guidance for best practices.

Using Roles

Roles do the heavy lifting, but playbooks are how you actually do work. The playbooks marry the inventory and the roles and specify what roles to play on which host. Here is what a playbook with roles looks like:

1 2 3 4 5 6 7

--- - hosts: all   roles:     - roles/common

Running the playbook produces the following output:

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

ansible-playbook -i hosts playbook_with_roles.yml --sudo PLAY *************************************************************************** TASK [setup] ******************************************************************* ok: [larry] ok: [moe] ok: [curly] TASK [roles/common : Create a user named gigi] ********************************* changed: [curly] changed: [moe] changed: [larry] TASK [roles/common : Add public key] ******************************************* changed: [larry] changed: [curly] changed: [moe] TASK [roles/common : Copy the .gigirc file to the home directory of the new user gigi] *** changed: [moe] changed: [larry] changed: [curly] TASK [roles/common : Install common packages] ********************************** changed: [curly] => (item=[u'vim', u'htop', u'tmux']) changed: [moe] => (item=[u'vim', u'htop', u'tmux']) changed: [larry] => (item=[u'vim', u'htop', u'tmux']) PLAY RECAP ********************************************************************* curly                      : ok=5    changed=4    unreachable=0    failed=0 larry                      : ok=5    changed=4    unreachable=0    failed=0 moe                        : ok=5    changed=4    unreachable=0    failed=0<br>

Conclusion

Ansible is a great tool. It is lightweight. It can be used interactively with ad-hoc commands, and it scales very well to massive systems. It also has a lot of momentum and a great community. If you manage or even just work with remote servers, you want Ansible.