This article provides information on installing Weka on a Bright cluster. There are two parts two this article, the first part provides information on setting up Weka’s stateless client functionality on a software image to be used with Bright cluster nodes, the second part of this article provides information on setting up Bright nodes as Weka storage nodes to be part of a Weka storage cluster.
Installation of the WekaIO agent on top of a Bright cluster (stateless clients)
This section of the article provides information on installing the Weka agent on a Bright software image in order to mount Weka filesystems. The following steps use Weka’s stateless client functionality, so when the first Weka filesystem is mounted on the standard nodes, the Weka agent installed as part of the software image will automatically install the other required components from the Weka servers.
Note that the following steps work when using an OS that uses systemd (such as RHEL/CentOS 7 and later). Older operating systems that do not use systemd (such as CentOS/RHEL 6) require manual configuration to autofs on the nodes rather than defining the mounts in cmsh/Bright View.
Chroot to the software image that you wish to install the Weka agent
Note that the cm-chroot-sw-img command should be used in this step as the Weka agent installation requires certain filesystems available such as /proc. These are mounted automatically when using cm-chroot-sw-img compared to running the raw chroot command manually. Use the “exit” command to exit out of the chroot environment.
# cm-chroot-sw-img /cm/images/<your-image-name>
# curl http://<your-weka-backend-ip-or-hostname>:14000/dist/v1/install | sh
# exitReboot the standard nodes or perform an imageupdate in order to sync the Weka agent changes across
Before mounting a Weka filesystem on the nodes, the changes performed in the previous step must first be synced to the standard nodes. There are a couple of ways of doing this, you can either reboot the nodes in order to trigger the node-installer provisioning during the node boot process to sync the changes to the node local disk, or you can also perform an imageupdate operation in order to update the running nodes. Information regarding imageupdate may be found in the Bright admin manual: https://support.brightcomputing.com/manuals/9.0/admin-manual.pdf#subsection.5.6.2
Define a Weka filesystem at the node or category level
The following example defines a “/wekafs” mount at the “default” category level:
# cmsh
% category use default
% fsmounts
% add /wekafs
% set device <your-weka-backend-ip-or-hostname>/my_weka_filesystem
% set filesystem wekafs
% set mountoptions x-systemd.requires=weka-agent.service,x-systemd.mount-timeout=infinity,_netdev
% commitCheck that the filesystem is mounted on a node
After around a minute, the weka filesystem should then be mounted on the node. You can check this by looking at the output of the “df -h” command on one of the nodes
# ssh <your-node-name>
# df -hAdd the Weka filesystem to the exclude lists
Since Bright 8.1-21/8.2-18/9.0-4, CMDaemon recognizes wekafs as an external filesystem, however it is still best practice to add the Weka filesystem to all the exclude lists to ensure that it is not touched and filesystem data not accidentally removed, for instance, when an imageupdate against a node is performed. For example, to add the filesystem to the exclude lists in the default category
# cmsh
% category use default
% set excludelistupdate
(add the mount to the exclude list then save and quit the editor)
% set excludelistsyncinstall
(add the mount to the exclude list then save and quit the editor)
% set excludelistfullinstall
(add the mount to the exclude list then save and quit the editor)
% set excludelistgrab
(add the mount to the exclude list then save and quit the editor)
% set excludelistgrabnew
(add the mount to the exclude list then save and quit the editor)
% commitInstallation of the WekaIO storage server on top of a Bright cluster (storage nodes on top of Bright)
Due to how the WekaIO installation currently works, there is currently no clean way of installing the Weka server components directly inside the software image in a chroot environment. In order to install the Weka components inside a software image, Weka must be installed directly on one of the nodes in the Bright cluster, then a node-to-image sync (grabimage) performed against the node that has Weka installed.
Download the WekaIO tar file
Download the WekaIO tar file on to one of the backend nodes you intend to be part of the cluster
Untar the WekaIO tar file and run the installer script
[root@node001 ~]# tar -xf weka-3.8.0.tar
[root@node001 ~]# cd weka-3.8.0
[root@node001 weka-3.8.0]# ./install.shClean up the installation files
Clean up by removing the installation directory and the tar file
[root@node001 weka-3.2.2]# cd ..
[root@node001 ~]# rm -rf weka-3.8.0 weka-3.8.0.tarCreate a new software image for Weka
On the head node, create a new software image for WekaIO based off the original software image for the node. For this example, we are going to assume that node001 currently uses the default-image and that we are going to name the new image: wekaio-image
[root@ah-c-02-22-b82-c7u5 ~]# cmsh
[ah-c-02-22-b82-c7u5]% softwareimage
[ah-c-02-22-b82-c7u5->softwareimage]% clone default-image wekaio-image
[ah-c-02-22-b82-c7u5->softwareimage*[wekaio-image*]]% commit
[ah-c-02-22-b82-c7u5->softwareimage[wekaio-image]]%Perform a node-to-image sync from the Weka node
Use the grabimage command in cmsh to grab the installation of WekaIO on the node and install it into the newly created image
[ah-c-02-22-b82-c7u5]% device
[ah-c-02-22-b82-c7u5->device]% grabimage -w -i wekaio-image node001Create a new Weka node category
Create a new node category based on the original category used for the node to be used for WekaIO in cmsh. We will point this node category to our new WekaIO software image. We will also need to set up exclusion lists to make sure Weka keeps it save-state information during node update/synchronization operations.
[ah-c-02-22-b82-c7u5]% category
[ah-c-02-22-b82-c7u5->category]% clone default wekaio
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% set softwareimage wekaio-image
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% set excludelistsyncinstall
(add the following lines to the file)
- /opt/weka/data/*
- /opt/weka/traces/*
- /opt/weka/support/*
- /opt/weka/analytics/*
- /opt/weka/events/*
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% set excludelistupdate
(add the following lines to the file)
- /opt/weka/data/*
- /opt/weka/traces/*
- /opt/weka/support/*
- /opt/weka/analytics/*
- /opt/weka/events/*
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% set excludelistgrab
(add the following line to the file)
- /opt/weka/data/*
- /opt/weka/traces/*
- /opt/weka/support/*
- /opt/weka/analytics/*
- /opt/weka/events/*
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% set excludelistgrabnew
(add the following line to the file)
- /opt/weka/data/*
- /opt/weka/traces/*
- /opt/weka/support/*
- /opt/weka/analytics/*
- /opt/weka/events/*
[ah-c-02-22-b82-c7u5->category*[wekaio*]]% commit
[ah-c-02-22-b82-c7u5->category[wekaio]]%Set the Weka storage nodes category to the newly created category
Before rebooting the nodes, we need to set the nodes you wish WekaIO to be installed on to use the newly created category to ensure they boot off the new Weka image and have the appropriate exclude lists set. For example to set nodes node001-node005 to use the WekaIO category:
[ah-c-02-22-b82-c7u5]% device
[ah-c-02-22-b82-c7u5->device]% foreach -c wekaio (reboot)
node001: Reboot in progress ...
node002: Reboot in progress ...
node003: Reboot in progress ...
node004: Reboot in progress ...
node005: Reboot in progress ...
[ah-c-02-22-b82-c7u5->device]%Check the Weka status on the nodes
Once the nodes are back online, check to make sure WekaIO is running
[ah-c-02-22-b82-c7u5]% device
[ah-c-02-22-b82-c7u5->device]% foreach -c wekaio (get hostname; ssh "weka local ps")
node001
Container | Running | Disabled | Monitoring | Persistent | PID | Stop Reason
----------+---------+----------+------------+------------+------+------------
default | True | False | True | True | 2072 |
node002
Container | Running | Disabled | Monitoring | Persistent | PID | Stop Reason
----------+---------+----------+------------+------------+------+------------
default | True | False | True | True | 2063 |
node003
Container | Running | Disabled | Monitoring | Persistent | PID | Stop Reason
----------+---------+----------+------------+------------+------+------------
default | True | False | True | True | 2066 |
[ah-c-02-22-b82-c7u5->device]%