How does the mlnx-ofed*-install.sh/qlgc-ofed-install.sh script work?

This article is being updated. Please be aware the content herein, not limited to version numbers and slight syntax changes, may not match the output from the most recent versions of Bright. This notation will be removed when the content has been updated.

The script installs “mlnx-ofa_kernel.x86_64” and “mlnx-ofa_kernel-devel.x86_64” on the head node or in the software image. These are the actual OFED stack packages that provide the kernel headers and modules. The script also installs the “mlnx-ofed2-modules.x86_64” package which provides the module files for the openmpi and mvapich2 packages installed by the OFED stack. For example:

# /cm/local/apps/mlnx-ofed2/current/bin/mlnx-ofed2-install.sh -s mlnx-test

The mlnx-ofed* packages just extracts the required RPMs and install scripts on the head node so they are not part of the OFED stack. The script will create a log file which contains a list of OFED package names that will be installed from the base distribution. For example:

# cat /root/cm/ofed/dist-ofed-rpms-softwareimage-mlnx-test.2014-04-07-10-32.log | xargs

compat-dapl.x86_64 compat-dapl-devel.x86_64 compat-dapl-utils.x86_64 compat-opensm-libs.x86_64 dapl.x86_64 dapl-devel.x86_64 dapl-utils.x86_64 ibutils.x86_64 ibutils-libs.x86_64 infiniband-diags.x86_64 libcxgb3.x86_64 libibcm.x86_64 libibcm.i686 libibcm-devel.i686 libibcm-devel.x86_64 libibcommon.x86_64 libibcommon.i686 libibcommon-devel.i686 libibcommon-devel.x86_64 libibmad.x86_64 libibmad.i686 libibmad-devel.i686 libibmad-devel.x86_64 libibumad.x86_64 libibumad.i686 libibumad-devel.x86_64 libibumad-devel.i686 libibverbs.x86_64 libibverbs.i686 libibverbs-devel.x86_64 libibverbs-devel.i686 libibverbs-utils.x86_64 libipathverbs.x86_64 libmlx4.x86_64 libmthca.x86_64 librdmacm.x86_64 librdmacm-devel.x86_64 librdmacm-utils.x86_64 opensm.x86_64 opensm-devel.i686 opensm-devel.x86_64 opensm-libs.x86_64 opensm-libs.i686 perftest.x86_64 qperf.x86_64 rdma.noarch rds-tools.x86_64

Notes:

Naming convention for the log file for head nodes should be:

 /root/cm/ofed/dist-ofed-rpms-headnode-<short hostname>.<date-time>.log

and for the software images:

/root/cm/ofed/dist-ofed-rpms-softwareimage-<software image name>.<date-time>.log

Hidden options for the mlnx-ofed*-install.sh script

The “-r” option can be used with mlnx-ofed*-install.sh to remove the installed OFED packages:

# /cm/local/apps/mlnx-ofed2/current/bin/mlnx-ofed2-install.sh -r -s default-image

Remove OFED stack RPMS, version: 2.0-2.6.9.6.g3a2d7bf.

The “-r” option removes all the installed OFED RPM packages. The log files under the directory /root/cm/ofed/ will have a list of the distribution packages which are removed from the software image:

# cat /root/cm/ofed/dist-ofed-rpms-softwareimage-mlnx.2014-03-14-12-48.log

Removing the packages from a head node or software image can lead to package dependency breakage, and software not working any more. So using the “-r” option should be done with caution.

Install the OFED packages provided from the base distribution

When the distribution rpm repositories are configured properly, the OFED packages of the base distribution can be installed with the following command:

# chroot /cm/images/mlnx-test yum install $(cat /root/cm/ofed/dist-ofed-rpms-softwareimage-mlnx-test.2014-04-07-10-32.log)