↓Skip to main content

Now Let's Automate Ansible

Author
Jeffrey Forman
Table of Contents

Over the past few years, I’ve been trying to make it easier to manage my homelab.

In order of implementation:

  1. Set up local services: DHCP, DNS, VLANs, etc.
  2. Standardize on Proxmox as a virtualization platform.
  3. Learn Ansible to manage the growing number of machines, physical and virtual.
  4. Populate my local instance of Netbox with IP ranges, addresses, and hardware information.

Throuhought, document it all on my local wiki.

But at some point, running Ansible over and over again to keep things up to date became both tedious and error probe. What happens when I want to run an Ansible playbook to automatically update Netbox with new IP address information from a VM? And I want to do it automatically.

Enter Semaphore. It’s basically a web UI that wraps crontab with the ability to execute actions such as Ansible playbooks, shell scripts, Terraform, etc.

This is my reminder, and hopefully a help for others, about how I tied all these technologies together.

Let’s go
#

First, install Semaphore. I spun up a VM and run Semaphore via Docker compose.

Semaphore’s website provides point-and-click help for writing the compose file. My final result:

services:
    semaphore:
        ports:
            - 3000:3000
        image: semaphoreui/semaphore:v2.11.2
        environment:
            SEMAPHORE_DB_DIALECT: bolt
            SEMAPHORE_ADMIN: admin
            SEMAPHORE_ADMIN_PASSWORD: initialpassword
            SEMAPHORE_ADMIN_NAME: Admin
            SEMAPHORE_ADMIN_EMAIL: someemail@domain.net
            SEMAPHORE_EMAIL_ALERT: True
            SEMAPHORE_EMAIL_SENDER: semaphore@domain.net
            SEMAPHORE_EMAIL_HOST: 192.168.1.1
            ANSIBLE_HOST_KEY_CHECKING: False
        volumes:
            - semaphore_data:/var/lib/semaphore
            - semaphore_config:/etc/semaphore
            - semaphore_tmp:/tmp/semaphore
volumes:
    semaphore_data:
    semaphore_config:
    semaphore_tmp:

Once Semaphore is running, wire up SSH keys and Git repositories for your Ansible configs. Given I use Netbox as my source of truth (Ansible’s inventory), I needed to use the Ansible inventory plugin for Netbox.

The key here is to put the entire Ansible YAML into the Inventory input box in Semaphore. Select Static YAML and insert the following, substituting your relevant endpoint and token data:

plugin: netbox.netbox.nb_inventory
validate_certs: true
api_endpoint: "https://netbox.your.domain.here/"
token: "YOURNETBOXTOKENHERE"
group_by:
  - tags
group_names_raw: true
query_filters:
  - cf_ansible_managed: true

Now when you try to run your first playbook, you might be greeted with the following error:

4:24:09 PM [WARNING]:  * Failed to parse
4:24:09 PM /tmp/semaphore/inventory_2147483642/inventories/netbox_inv.yaml with auto
4:24:09 PM plugin: pytz must be installed to use this plugin
4:24:09 PM [WARNING]:  * Failed to parse
4:24:09 PM /tmp/semaphore/inventory_2147483642/inventories/netbox_inv.yaml with yaml
4:24:09 PM plugin: Plugin configuration YAML file, not YAML inventory
4:24:09 PM [WARNING]:  * Failed to parse
4:24:09 PM /tmp/semaphore/inventory_2147483642/inventories/netbox_inv.yaml with ini
4:24:09 PM plugin: Invalid host pattern '---' supplied, '---' is normally a sign this is a
4:24:09 PM YAML file.
4:24:09 PM [WARNING]: Unable to parse
4:24:09 PM /tmp/semaphore/inventory_2147483642/inventories/netbox_inv.yaml as an inventory
4:24:09 PM source
4:24:09 PM [WARNING]: No inventory was parsed, only implicit localhost is available
4:24:09 PM [WARNING]: provided hosts list is empty, only localhost is available. Note that
4:24:09 PM the implicit localhost does not match 'all'

Sifting through the output, the most relevant line is

4:24:09 PM plugin: pytz must be installed to use this plugin

What is pytz? A timezone calculation library.

It’s unfortunate Semaphore marks this as a WARNING and not an ERROR in the Semaphore UI. Why? Because no hosts are selected as a result of being unable to process the inventory, due to the lack of pytz, yet Sempahore marks this as a SUCCESS.

What is this error? Ansible interacting with Netbox requires the pytz Python package as part of its installation. pytz is Python’s library to perform timezone calculations. But how do you add pytz to the installation when you don’t necessarily have control over how Ansible is built within Semaphore? Enter requirements.txt integration. Integration with the Docker compose method of running the app is not documented here, but I found further explanation in a Semaphore Github discussion related to Ansible.

I needed to add to my docker-compose yaml:

services:
    sempahore:
        volumes:
            - /home/casket/docker/etc/requirements.txt:/etc/semaphore/requirements.txt

Where the contents of /home/casket/docker/etc/requirements.txt are

pytz

This tells Semaphore, that upon startup of the application, add the pytz library.

Victory.

Now to try to figure out how to get Docker containers to work with IPv6…

Sources
#