Managing software images with Puppet

Managing software images with puppet provides a number of advantages when it comes to automation and revision management. Coupling puppet with a version control system (git for example) may improve the software image workflow and lifecycle.

This FAQ will use the roles and profiles example from the Puppetlabs website. We will build a Jenkins server image.
An installation of Puppet 5 on a head node is required, please see the Puppetlabs documentation for installation.

Best Practice

Best practice for deploying puppet managed software images is to clone the default-image generated by the Bright installation and treat each image as having a role with profiles attached.

For example.

Each image may correlate to a type of compute node and be broken down by function.
Cpu-only or GPU compute node. At the same time, you may want to have a workload manager profile

In Puppet we would consider the role of a system as a compute node with a profile of CPU-only or GPU.
It may also have the profile of slurm as the workload manager.

Another role in Puppet might be a storage node with a profile of Ceph or ZFS.

Step 1 : Puppet server installation

Generally, installation involves adding a repository to you package manager (yum/dnf/zypper/apt) and installing the required package. Once puppet is installed, create the roles and profiles modules as described in the puppet labs example. Finally start the puppet server daemon.

Step 2 : Configure puppet roles and modules

For the roles module: /etc/puppetlabs/code/environments/production/modules/roles/manifests/jenkins/master.pp
class roles::jenkins::master {
  include profiles::jenkins::master
}

For the profiles module: /etc/puppetlabs/code/environments/production/modules/profiles/manifests/jenkins/master.pp
class profiles::jenkins::master (
  String $jenkins_port = '9091',
  String $java_dist    = 'jdk',
  String $java_version = 'latest',
) {
  class { 'jenkins':
    configure_firewall => false,
    install_java       => false,
    port               => $jenkins_port,
    config_hash        => {
      'HTTP_PORT'    => { 'value' => $jenkins_port },
      'JENKINS_PORT' => { 'value' => $jenkins_port },
    },
  }
  class { 'java':
    distribution => $java_dist,
    version      => $java_version,
    before       => Class['jenkins'],
  }
}

With the roles and profiles modules created, create a site file in /etc/puppetlabs/code/environments/production/manifests/site.pp with contents:
node jenkins {
    include roles::jenkins::master
}

The above code will apply the Jenkins master class to a node named jenkins.

Step 3: Jenkins puppet module

You will need to install the Jenkins puppet module onto the puppet server

# puppet module install rtyler/jenkins

Step 4: Create a base software image

 Next we will need to build a software image for puppet to manage. This is done using the standard tools in Bright.

# cmsh -c "softwareimage; clone default-image jenkins-image; commit"

This will give us a image called jenkins-image. We will use this image to install jenkins.

Step 5: Chroot into the software image, install and run puppet

 By default, a chroot environment inherits the hostname of the system. Currently puppet uses the system hostname to determine the resources to apply. There is a utility called chname, which allows a chroot environment to be created with a different hostname. See https://github.com/marineam/chname.

In order for the puppet agent to run in the chroot, you will need to bind mount /proc and /sys.

# mount -o bind /proc /cm/image/jenkins-image/proc
# mount -o bind /sys /cm/image/jenkins-image/sys
# chname jenkins chroot /cm/images/jenkins-image /bin/bash

install the puppet agent into the software image as per the puppet installation documentation.

For Redhat and CentOS system:

# yum --installroot=/cm/images/jenkins-image install puppet-agent

Run the puppet agent inside the chroot. Puppet will configure jenkins packages inside the image.

# puppet agent --test

Once complete, the image may be deployed to a compute node using the standard Bright tools.

Caveats

Where possible avoid using Puppet service management resources to start and stop services. Enabling and disabling services is not an issue. In the chroot environment, puppet will actually start a service on the real head node if instructed to do so.