Automating execution of SCM-managed Ansible playbooks

Have you ever wanted to automate execution of certain tasks? Then you have probably encountered Ansible. But have you considered how can you automate Ansible execution even further?

Let’s imagine that you have a set of tasks to perform. Let’s call them A, B and C for simplicity. Let’s also suppose that you can execute these tasks using Ansible on some Linux host.

You write a playbook that carries out the aforementioned jobs and execute it using CLI. Everything works and you are happy. But what if you want to periodically trigger this playbook and allow other programs to start it? Or how do you allow developers to change the playbook code easily? Today we will demonstrate one way to achieve these goals by automating Ansible even further – by exposing SCM-managed playbooks via API endpoints.

To accomplish this assignment, we will use the following technologies:

• GitLab – our SCM; here we store playbooks (yaml files),
• Ansible AWX – our Ansible execution environment; allows for execution of playbooks triggered by API calls.

We assume that the reader has some basic knowledge about the aforementioned technologies.

Setting up execution enviornment

First we need to install all requirements for Ansible AWX. In this tutorial we will use Minikube for Kubernetes environment, Docker as the container runtime and Kustomize to help us with AWX installation.

Installation guides for the aforementioned technologies can be found below:

Docker:

https://docs.docker.com/engine/install/ubuntu/

Minikube:

https://minikube.sigs.k8s.io/docs/start/

Kustomize:

https://kubectl.docs.kubernetes.io/installation/kustomize/

Since the official guide for basic Ansible AWX installation uses the same set of technologies as proposed in this article, we will not quote the aforementioned guide and leave it to the reader to install the AWX (official guide can be found here: https://github.com/ansible/awx-operator#basic-install). 

One note the consider is to watch out for system requirements for Ansible AWX. Theoretically, one can install AWX on a machine with insufficient resources but this may result in error loop during last steps of installation process.

Once AWX is installed, one can get the Minikube service associated with it by running:

minikube service awx-demo-service --url -n $NAMESPACE

And then we get the admin credentials for AWX:

kubectl get secret awx-demo-admin-password -o jsonpath="{.data.password}" | base64 --decode

Now we can go to the url returned from the “minikube service” command and log in with credentials returned from the “kubectl get secret” command.

Once we access Ansible AWX, we can start configuring our automation.

Setting up Ansible AWX

To allow users to trigger SCM-managed playbooks via API calls we need to configure the following items:

• Organization,
• Inventory and hosts,
• Credential,
• Project,
• Job template.

We start by configuring Organization which will set the scope of created objects.

In the left panel go to Access -> Organisations and click “Add”. 

New window with Organization configuration appears. We will only input the name of our choice and leave the rest of the options with default values.

Next, we will configure Inventory which will define which host should execute our playbook. In the left panel go to Resources -> Inventories and click “Add Inventory”.

New window with Inventory configuration appears. Here we choose a name for our Inventory and choose the Organization that we created in the previous step. This way, this Inventory will only be accessible in the given Organization.

Then, we go back to our newly created object and on the top navbar choose “Hosts” and click “Add”. Here you can add hosts as if you were adding them directly in the playbook.

Next, we will configure Credential for SCM. We will use private and public keys for this purpose, so first we need to generate them. We can do so by running the following command on a Linux host:

ssh-keygen -f awx_ssh_key

After we generated the keys, we can continue with AWX configuration.

In the left panel go to Resources -> Credentials and click “Add” and configure the Credential as follows:

• Organization – name of the previously created Organization object,
• Credential type – Source Control,
• SCM Private key – contents of private key generated in the previous step,
• Private key passphrase – password for the private key.

The last thing to do with the keys is to upload the public key to SCM (GitLab) and add it to the account. Once it is done, AWX can authenticate using private key.

Next step is to configure Project. Project will be dedicated to the given Organization and will allow us to download playbooks from SCM (using Credential) and execute them via Job Templates on defined hosts (using Inventory and Hosts).

In the left panel, go to Resources -> Projects and click “Add”.

New window with configuration appears. Here we need to provide a name for our project, choose the appropriate Organization and choose “Source Control Type” as “Git”. In the section “Source Details” we need to provide details about the SCM repo (like url and Credential created previously).

The last thing to configure is Job Template. It will allow us to perform certain playbook from SCM-managed project via API call.

In the left panel, go to Resources -> Templates and click “Add job template”.

New configuration window appears. Here we set the name of the template and then set configuration fields as follows:

• Job Type – set to “Run”,
• Inventory – name of previously created object,
• Project – name of previously created object,
• Playbook – set to playbook of your choice that is stored in the SCM (note that AWX automatically syncs available playbooks from the given repo).

After finishing the last step of configuration, we end up with AWX fully equipped with playbooks which are SCM-managed – meaning that programmers can change them on the go as they work and Ansible AWX will automatically sync changes whenever the job is ran.

However, the second question is still to be answered – how do we programatically start this job? For this purpose, we will use API calls.

Ansible AWX comes with API calls enabled by default. What we need, is to authenticate and send a single request to start the job.

We will authenticate using “basic” method. We can start job by sending a POST request to the following url:

https://awx.server.com/api/v2/job_templates//launch/

The response contains jobId of our request. We can check the status by sending GET request to the following url:

https://awx.server.com/api/v2/jobs//

All job triggered by API calls are asynchronous. So what is left, is to periodically check the status of our job to ensure proper execution.

That is it! We have shown how you can easily connect SCM with AWX and allow for machine-to-machine communication via API calls. Hopefully, the reader will find this information useful.