Managing software images with Ansible is sometimes tricky as software images live inside their own chroot directory.
To make managing software images easier, our brightcomputing.bcm
1 Ansible collection contains two roles that transparently integrate software images into your Ansible inventory.
brightcomputing.bcm.software_image_tunnels_open
brightcomputing.bcm.software_image_tunnels_close
With these roles, you can manage the content of software images as if they were regular, SSH-accessible, inventory hosts. This allows you to use the same playbooks/roles to manage physical nodes, software images, or both.
1 See “Section 17.10: Ansible” of the Bright Cluster Manager 9.1: Administrator Manual for more on the brightcomputing.bcm
collection
Overview
The above-mentioned two Ansible roles use the SSH server’s ChrootDirectory
option to bind a non-standard SSH port to the inside of a software image directory. Then using the SSH client’s ProxyCommand
option, Ansible will SSH to the head node, and tunnel to 127.0.0.1
on the non-standard SSH port, rooting Ansible inside the respective software image chroot directory.
Installation
$ ansible-galaxy collection install brightcomputing.bcm
You need a minimum version 9.1.10+ to use the brightcomputing.bcm.software_image_tunnels_*
roles.
$ ansible-galaxy collection list brightcomputing.bcm
Collection Version
------------------- -----------------------
brightcomputing.bcm 9.1.10+41573.git448b052
Configuration
The brightcomputing.bcm.software_image_tunnels_*
roles require an Ansible inventory group named software_images
2, with a specially crafted ansible_ssh_common_args
variable in your inventory.yml
.
Within this group, define each software image with its own ansible_port
. The port numbers can be arbitrary but MUST be unique and available.
---
# inventory.yml
all:
vars:
ansible_user: root
hosts:
head_node:
ansible_host: 1.2.3.4
ansible_python_interpreter: /cm/local/apps/python3/bin/python
software_images:
vars:
ansible_ssh_common_args: '-o StrictHostKeyChecking=no -o ProxyCommand="ssh -q -W 127.0.0.1:%p {{ hostvars.head_node.ansible_user }}@{{ hostvars.head_node.ansible_host }}"'
hosts:
default-image:
ansible_port: 2222
my-image:
ansible_port: 2223
2 The inventory group name is configurable. See Use a custom inventory group name
In the above inventory.yml
, there are 2 example software images defined:
default-image
on port 2222my-image
on port 2223
The key configuration is the ProxyCommand
in the ansible_ssh_common_args
of the software_images
:
ssh -q -W 127.0.0.1:%p {{ hostvars.head_node.ansible_user }}@{{ hostvars.head_node.ansible_host }}
This instructs Ansible to quietly (-q
) redirect stdin/stdout (-W
) to 127.0.0.1:%p
3 after connecting to the head node.
default-image: ssh -q -W 127.0.0.1:2222 root@1.2.3.4
my-image: ssh -q -W 127.0.0.1:2223 root@1.2.3.4
3%p
is replaced by the respective ansible_port
at play execution time
Usage
To run plays targeting your software images, first open the software image tunnels, exposing your software images as inventory hosts. After managing your software images, be sure to close the software image tunnels.
The general workflow is summarized as follows:
- Open the software image tunnels.
- Manage your software image using standard Ansible best practices.
- Close the software image tunnels.
Open the software image tunnels
Apply the brightcomputing.bcm.software_image_tunnels_open role to your head node.
---
# open-tunnels.yml
- name: Open Software Image Tunnels
hosts: head_node
gather_facts: false
roles:
- brightcomputing.bcm.software_image_tunnels_open
Manage your software images
Apply any Ansible roles
or tasks
, just as you would any other host4.
---
# manage-software-images.yml
- name: Manage Software Images
hosts: software_images
gather_facts: false
tasks:
- ping:
4 See Caveats for some systemd
restrictions
Close the software image tunnels
Apply the brightcomputing.bcm.software_image_tunnels_close role to your head node.
---
# close-tunnels.yml
- name: Close Software Image Tunnels
hosts: head_node
gather_facts: false
roles:
- brightcomputing.bcm.software_image_tunnels_close
Putting it all together
Development
We recommend using individual playbooks and running individual ansible-playbook
commands to speed up development, as you do not repeatedly open and close the software image tunnels.
# once at the beginning of development
$ ansible-playbook -i inventory.yml open-tunnels.yml
# multiple executions during development
$ ansible-playbook -i inventory.yml ping-software-images.yml
# once at the end of development
$ ansible-playbook -i inventory.yml close-tunnels.yml
Production
For production executions, you can create one playbook by combining the above with import_playbook
:
---
# playbook.yml
- import_playbook: open-tunnels.yml
- import_playbook: configure-software-images.yml
- import_playbook: close-tunnels.yml
This opens, manages and closes the tunnels all in one command:
$ ansible-playbook -i inventory.yml playbook.yml
Example
$ ansible-playbook -i inventory.yml playbook.yml
PLAY [Open Software Image Tunnels on Head Node]
TASK [brightcomputing.bcm.software_image_tunnels : Gather distro specific facts]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Include distro specific variables]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Ensure Port configuration exists in /etc/ssh/sshd_config]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Open SSH tunnels]
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/open-tunnel.yml for head_node => (item=default-image)
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/open-tunnel.yml for head_node => (item=my-image)
TASK [brightcomputing.bcm.software_image_tunnels : Mount pseudo filesystems under /cm/images/default-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)
TASK [brightcomputing.bcm.software_image_tunnels : Ensure pseudo systemd-resolvconf run directory exists in chroot]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Copy host's resolv.conf to chroot]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Listen on SSH port 2222]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Bind SSH port 2222 to chroot /cm/images/default-image]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Mount pseudo filesystems under /cm/images/my-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)
TASK [brightcomputing.bcm.software_image_tunnels : Ensure pseudo systemd-resolvconf run directory exists in chroot]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Copy host's resolv.conf to chroot]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Listen on SSH port 2223]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Bind SSH port 2223 to chroot /cm/images/my-image]
changed: [head_node]
RUNNING HANDLER [brightcomputing.bcm.software_image_tunnels : Restart ssh]
changed: [head_node]
***
PLAY [Ping Software Image Inventory Hosts]
TASK [ping]
ok: [my-image]
ok: [default-image]
***
PLAY [Close Software Image Tunnels on Head Node]
TASK [brightcomputing.bcm.software_image_tunnels : Gather distro specific facts]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Include distro specific variables]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Ensure Port configuration exists in /etc/ssh/sshd_config]
ok: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Close SSH tunnels]
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/close-tunnel.yml for head_node => (item=default-image)
included: brightcomputing/bcm/roles/software_image_tunnels/tasks/close-tunnel.yml for head_node => (item=my-image)
TASK [brightcomputing.bcm.software_image_tunnels : Unbind SSH port 2222 from chroot /cm/images/default-image]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Close SSH port 2222]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Unount pseudo filesystems under /cm/images/default-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)
TASK [brightcomputing.bcm.software_image_tunnels : Remove host's resolv.conf from chroot]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Remove /etc/fstab.default-image (temporary chroot fstab)]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Unbind SSH port 2223 from chroot /cm/images/my-image]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Close SSH port 2223]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Unount pseudo filesystems under /cm/images/my-image]
changed: [head_node] => (item=/dev)
changed: [head_node] => (item=/proc)
changed: [head_node] => (item=/sys)
TASK [brightcomputing.bcm.software_image_tunnels : Remove host's resolv.conf from chroot]
changed: [head_node]
TASK [brightcomputing.bcm.software_image_tunnels : Remove /etc/fstab.my-image (temporary chroot fstab)]
changed: [head_node]
RUNNING HANDLER [brightcomputing.bcm.software_image_tunnels : Restart ssh]
changed: [head_node]
PLAY RECAP
default-image : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
head_node : ok=32 changed=20 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
my-image : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Caveats
As a runningsystemd
is not available inside the software image chroot directory, Ansible modules that require a functioning systemd
do not fully work as expected.
For example, when using ansible.builtin.service
with software images, you CAN enable/disable services, but you CANNOT start/stop services.
Use a custom inventory group name
If for some reason you don’t want to, or can’t, use the inventory group name software_images
, you can use a different, custom group name instead.
This can be done by setting software_image_group_name
on the brightcomputing.bcm.software_image_tunnels_*
roles
---
# inventory.yml
# ...
my_software_images:
vars:
ansible_ssh_common_args: ...
hosts:
default-image:
ansible_port: 2222
my-image:
ansible_port: 2223
---
# open-tunnels.yml
- name: Open Software Image Tunnels
hosts: head_node
gather_facts: false
roles:
- role: brightcomputing.bcm.software_image_tunnels_open
vars: { software_image_group_name: my_software_images }
---
# manage-software-images.yml
- name: Manage Software Images
hosts: my_software_images
gather_facts: false
tasks:
- ping:
---
# close-tunnels.yml
- name: Close Software Image Tunnels
hosts: head_node
gather_facts: false
roles:
- role: brightcomputing.bcm.software_image_tunnels_open
vars: { software_image_group_name: my_software_images }