1. Home
  2. Enabling Lustre clients on BC 3

Enabling Lustre clients on BC 3

Following document walks through the process of installing and configuring lustre clients in BC 3

1. Installing Packages

This is the first step in preparing the BC3.0 cluster to obtain required client packages.


On some systems you may be required to install following packages

$ apt install libmount-dev libjson-c-dev

1.1​  Installing Lustre Packages on compute and login nodes.

 The below steps will guide through the process of installing luster package to the software image which would guarantee that the packages persist on the nodes across reboots, sync update or a full install.

  1. Copy existing packages to target directory in selected image. In this example we will be using default-image
cp  lustre-client-utils_2.14.0-ddn54-1_amd64.deb /cm/images/default-image/tmpcp  lustre-client-modules-dkms_2.14.0-ddn54-1_amd64.deb /cm/images/default-image/tmp
  1. Use cm-chroot-sw-image command to confirm packages are available in the image

cm-chroot-sw-img  /cm/images/default-image
cd /tmp

  1.  Reboot a node that is currently using default-image. Post reboot lustre packages will be available in /tmp on the host. Change to /tmp directory and install packages.
apt install ./lustre-client-utils_2.14.0-ddn54-1_amd64.deb ./lustre-client-modules-dkms_2.14.0-ddn54-1_amd64.deb libmount-dev libjson-c-dev
  1. Modprobe lustre and lnet to confirm required components are installed.
    modprobe lustre
  1. Exit to commit changes to software image 
  1. Repeat the above steps to install the packages on the required images (eg- dgx-osimage).

​1.2 Configure lustre on slogin and compute nodes.

Configure lustre nets after booting the node with the new software image from above step.

  1. Update the lustre.conf file on the cpu and compute nodes to reflect the below settings.

“o2ib0(<ib_interface1>,<ib_interface2>)”, ib_interface1 and ib_ interface2 are the two IB ports connected to the storage IB fabric.

The below example lists the configuration on client node with IB interfaces ibp97s0f0, ibp225s0f0 connected to storage

$ cat /etc/modprobe.d/lustre.conf
options mds mds_num_threads=256
options oss oss_num_threads=256
options libcfs cpu_npartitions=32
options libcfs cpu_pattern=””
options ksocklnd peer_credits=128
options ko2iblnd peer_credits=32
options lnet networks=”o2ib0(ibp97s0f0,ibp225s0f0)”

Step 4: Mount lustre at predetermined mount point .


  mount -t lustre,,,, /lustre


Step 5: Grab this software image to be deployed to rest of the hosts in the same category


  grabimage hostname   -w  -i  default-image

Repeat the above process for nodes with different architecture (eg- dgx nodes)

Follow instructions from Bright Admin manual for  adding this FS mounts to respective node categories.

Updated on December 1, 2022

Leave a Comment