1. Home
  2. Software Management
  3. Managing software images as Ansible inventory hosts

Managing software images as Ansible inventory hosts

Managing software images with Ansible is sometimes tricky as software images live inside their own chroot directory.

To make managing software images easier, our brightcomputing.bcm1 Ansible collection contains two roles that transparently integrate software images into your Ansible inventory.

  • brightcomputing.bcm.software_image_tunnels_open
  • brightcomputing.bcm.software_image_tunnels_close

With these roles, you can manage the content of software images as if they were regular, SSH-accessible, inventory hosts. This allows you to use the same playbooks/roles to manage physical nodes, software images, or both.

1 See “Section 17.10: Ansible” of the Bright Cluster Manager 9.1: Administrator Manual for more on the brightcomputing.bcm collection

Overview

The above-mentioned two Ansible roles use the SSH server’s ChrootDirectory option to bind a non-standard SSH port to the inside of a software image directory. Then using the SSH client’s ProxyCommand option, Ansible will SSH to the head node, and tunnel to 127.0.0.1 on the non-standard SSH port, rooting Ansible inside the respective software image chroot directory.

Installation

$ ansible-galaxy collection install brightcomputing.bcm

You need a minimum version 9.1.10+ to use the brightcomputing.bcm.software_image_tunnels_* roles.

$ ansible-galaxy collection list brightcomputing.bcm

Collection          Version               
------------------- -----------------------
brightcomputing.bcm 9.1.10+41573.git448b052

Configuration

The brightcomputing.bcm.software_image_tunnels_* roles require an Ansible inventory group named software_images2, with a specially crafted ansible_ssh_common_args variable in your inventory.yml.

Within this group, define each software image with its own ansible_port. The port numbers can be arbitrary but MUST be unique and available.

---
# inventory.yml

all:
  vars:
    ansible_user: root
  hosts:
    head_node:
      ansible_host: 1.2.3.4
      ansible_python_interpreter: /cm/local/apps/python3/bin/python

software_images:
  vars:
    ansible_ssh_common_args: '-o StrictHostKeyChecking=no -o ProxyCommand="ssh -q -W 127.0.0.1:%p {{ hostvars.head_node.ansible_user }}@{{ hostvars.head_node.ansible_host }}"'
  hosts:
      default-image:
        ansible_port: 2222
      my-image:
        ansible_port: 2223

2 The inventory group name is configurable. See Use a custom inventory group name

In the above inventory.yml, there are 2 example software images defined:

  • default-image on port 2222
  • my-image on port 2223

The key configuration is the ProxyCommand in the ansible_ssh_common_args of the software_images:

ssh -q -W 127.0.0.1:%p {{ hostvars.head_node.ansible_user }}@{{ hostvars.head_node.ansible_host }}

This instructs Ansible to quietly (-q) redirect stdin/stdout (-W) to 127.0.0.1:%p3 after connecting to the head node.

default-image:  ssh -q -W 127.0.0.1:2222 root@1.2.3.4
     my-image:  ssh -q -W 127.0.0.1:2223 root@1.2.3.4

3%p is replaced by the respective ansible_port at play execution time

Usage

To run plays targeting your software images, first open the software image tunnels, exposing your software images as inventory hosts. After managing your software images, be sure to close the software image tunnels.

The general workflow is summarized as follows:

  1. Open the software image tunnels.
  2. Manage your software image using standard Ansible best practices.
  3. Close the software image tunnels.

Open the software image tunnels

Apply the brightcomputing.bcm.software_image_tunnels_open role to your head node.

---
# open-tunnels.yml

- name: Open Software Image Tunnels
  hosts: head_node
  gather_facts: false
  roles:
    - brightcomputing.bcm.software_image_tunnels_open

Manage your software images

Apply any Ansible roles or tasks, just as you would any other host4.

---
# manage-software-images.yml

- name: Manage Software Images
  hosts: software_images
  gather_facts: false
  tasks:
    - ping:

4 See Caveats for some systemd restrictions

Close the software image tunnels

Apply the brightcomputing.bcm.software_image_tunnels_close role to your head node.

---
# close-tunnels.yml

- name: Close Software Image Tunnels
  hosts: head_node
  gather_facts: false
  roles:
    - brightcomputing.bcm.software_image_tunnels_close

Putting it all together

Development

We recommend using individual playbooks and running individual ansible-playbook commands to speed up development, as you do not repeatedly open and close the software image tunnels.

# once at the beginning of development
$ ansible-playbook -i inventory.yml open-tunnels.yml

# multiple executions during development
$ ansible-playbook -i inventory.yml ping-software-images.yml

# once at the end of development
$ ansible-playbook -i inventory.yml close-tunnels.yml

Production

For production executions, you can create one playbook by combining the above with import_playbook:

---
# playbook.yml

- import_playbook: open-tunnels.yml
- import_playbook: configure-software-images.yml
- import_playbook: close-tunnels.yml

This opens, manages and closes the tunnels all in one command:

$ ansible-playbook -i inventory.yml playbook.yml

Example

$ ansible-playbook -i inventory.yml playbook.yml

PLAY [Open Software Image Tunnels on Head Node]

TASK [brightcomputing.bcm.software_image_tunnels : Gather distro specific facts]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Include distro specific variables]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Ensure Port configuration exists in /etc/ssh/sshd_config]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Open SSH tunnels]
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/open-tunnel.yml for head_node => (item=default-image)
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/open-tunnel.yml for head_node => (item=my-image)

TASK [brightcomputing.bcm.software_image_tunnels : Mount pseudo filesystems under /cm/images/default-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)

TASK [brightcomputing.bcm.software_image_tunnels : Ensure pseudo systemd-resolvconf run directory exists in chroot]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Copy host's resolv.conf to chroot]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Listen on SSH port 2222]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Bind SSH port 2222 to chroot /cm/images/default-image]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Mount pseudo filesystems under /cm/images/my-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)

TASK [brightcomputing.bcm.software_image_tunnels : Ensure pseudo systemd-resolvconf run directory exists in chroot]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Copy host's resolv.conf to chroot]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Listen on SSH port 2223]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Bind SSH port 2223 to chroot /cm/images/my-image]
changed: [head_node]

RUNNING HANDLER [brightcomputing.bcm.software_image_tunnels : Restart ssh]
changed: [head_node]

***

PLAY [Ping Software Image Inventory Hosts]

TASK [ping]
ok: [my-image]
ok: [default-image]

***

PLAY [Close Software Image Tunnels on Head Node]

TASK [brightcomputing.bcm.software_image_tunnels : Gather distro specific facts]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Include distro specific variables]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Ensure Port configuration exists in /etc/ssh/sshd_config]
ok: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Close SSH tunnels]
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/close-tunnel.yml for head_node => (item=default-image)
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/close-tunnel.yml for head_node => (item=my-image)

TASK [brightcomputing.bcm.software_image_tunnels : Unbind SSH port 2222 from chroot /cm/images/default-image]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Close SSH port 2222]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Unount pseudo filesystems under /cm/images/default-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)

TASK [brightcomputing.bcm.software_image_tunnels : Remove host's resolv.conf from chroot]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Remove /etc/fstab.default-image (temporary chroot fstab)]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Unbind SSH port 2223 from chroot /cm/images/my-image]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Close SSH port 2223]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Unount pseudo filesystems under /cm/images/my-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)

TASK [brightcomputing.bcm.software_image_tunnels : Remove host's resolv.conf from chroot]
changed: [head_node]

TASK [brightcomputing.bcm.software_image_tunnels : Remove /etc/fstab.my-image (temporary chroot fstab)]
changed: [head_node]

RUNNING HANDLER [brightcomputing.bcm.software_image_tunnels : Restart ssh]
changed: [head_node]

PLAY RECAP
default-image              : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
head_node                  : ok=32   changed=20   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
my-image                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Caveats

As a runningsystemd is not available inside the software image chroot directory, Ansible modules that require a functioning systemd do not fully work as expected.

For example, when using ansible.builtin.service with software images, you CAN enable/disable services, but you CANNOT start/stop services.

Use a custom inventory group name

If for some reason you don’t want to, or can’t, use the inventory group name software_images, you can use a different, custom group name instead.

This can be done by setting software_image_group_name on the brightcomputing.bcm.software_image_tunnels_* roles

---
# inventory.yml

# ...

my_software_images:
  vars:
    ansible_ssh_common_args: ...
  hosts:
      default-image:
        ansible_port: 2222
      my-image:
        ansible_port: 2223
---
# open-tunnels.yml

- name: Open Software Image Tunnels
  hosts: head_node
  gather_facts: false
  roles:
    - role: brightcomputing.bcm.software_image_tunnels_open
      vars: { software_image_group_name: my_software_images }
---
# manage-software-images.yml

- name: Manage Software Images
  hosts: my_software_images
  gather_facts: false
  tasks:
    - ping:
---
# close-tunnels.yml

- name: Close Software Image Tunnels
  hosts: head_node
  gather_facts: false
  roles:
    - role: brightcomputing.bcm.software_image_tunnels_open
      vars: { software_image_group_name: my_software_images }
Updated on December 8, 2023

Related Articles

Leave a Comment