Categories

ID #1452

How do I upgrade my cluster if I have a storage node that is being shared?

 

How do I update a cluster with a storage node that is shared?

 

Upgrading with a Storage Node 

Special considerations must be made when upgrading Bright versions with a storage node internal to the cluster:

The problem with this kind of setup is that all of the nodes except for the head node(s) must be shut down before starting the upgrade.

But filesystems from the storage node are being shared, and they need to be available to the head node for the upgrade.

To solve this, we will temporarily get rid of the dependency of the storage nodes internal to the cluster.

Let's do it by example.

For this example, we use node003 (10.141.0.3) as a storage node. This storage node, when it is running, exports /cm/shared to the cluster from the local path of node003 at

/storage/cm/shared:

Steps:

1. Shut down all of the nodes, except for the head node(s), and the storage node.

In this case, assuming we have nodes node001 and node002 as the other nodes on the cluster, we will shut down those nodes:

% cmsh

[head]% device

[head->device]% shutdown node001 

[head->device]% shutdown node002

 

2. On the active head node, copy /cm/shared over to /cm/shared.local:

# mkdir –p /cm/shared.local

# cp –a /cm/shared/* /cm/shared.local


3. Remove the mount from Bright Cluster Manager, and then unmount the NFS filesystem /cm/shared:

Remove mount from Bright:

# cmsh

[head]% device use master

[head->device[head]]% fsmounts

[head->device[head]->fsmounts]% remove /cm/shared

[head->device[head]->fsmounts*]% commit

[head->device[head]->fsmounts]% quit

# umount /cm/shared

  

4. Bind-mount /cm/shared.local on /cm/shared

# mount --bind /cm/shared.local /cm/shared

To make it persistent, add the following line in /etc/fstab, after the autogenerated section:

# END AUTOGENERATED SECTION   -- DO NOT REMOVE

/cm/shared.local   /cm/shared   none  bind  0 0

set storage node install mode to “” if it is currently set to NOSYNC

# cmsh

[head]% device use node003

[head->device[node003]% set installmode ""

[head->device[node003*]% commit

 

5. Shut down your storage node.

# cmsh

[head]% device

[head->device]% shutdown node003

 

6. Perform the upgrade according to the instructions in this article:

/faq/index.php?action=artikel&cat=2&id=449

 

7. Boot your storage node.


8. Mount the storage node’s shared filesystem, /cm/shared,  onto /cm/shared.storage on the headnode.

Check  /etc/exports on node003 to make sure the filesystem is exported with rw permissions for the head node that will access it

On the head node create /cm/shared.storage:

# mkdir –p /cm/shared.storage
The filesystem is added to fstab, after the autogenerated section:

10.141.0.3:/storage/cm/shared    /cm/shared.storage    nfs    defaults   0 0

Mount the filesystem according to the fstab settings:

# mount -a 

 

9. Synchronize /cm/shared.storage and /cm/shared.local

# rsync -avh /cm/shared.local/ /cm/shared.storage/

 

10. Now unmount the /cm/shared bind mount:

# umount /cm/shared

Remove it from /etc/fstab on the head node. Ie remove the entry for /cm/shared.local:

/cm/shared.local   /cm/shared   none  bind  0 0

Remove this entry in /etc/fstab on the head node too:

10.141.0.3:/storage/cm/shared    /cm/shared.storage   nfs   defaults     0 0

 

11. Mount /cm/shared from the storage node to all the other nodes:

Using cmsh on the head node, mounting /cm/shared for the head node:

# cmsh

[head]% device use master

[head->device[head]% fsmounts

[head->device[head]->fsmounts% add /cm/shared

[head->device*[head*]->fsmounts*[/cm/shared*]% set device 10.141.0.3:/storage/cm/shared

[head->device*[head*]->fsmounts*[/cm/shared*]% set filesystem nfs

[head->device*[head*]->fsmounts*[/cm/shared*]% commit 

Using cmsh on the head node, mounting /cm/shared for the regular nodes:

# cmsh

[head->category[default]% category use default

[head->category[default]% fsmounts

[head->category[default]->fsmounts]% add /cm/shared

[head->category*[default*]->fsmounts*[/cm/shared*]% set device 10.141.0.3:/storage/cm/shared

[head->category*[default*]->fsmounts*[/cm/shared*]% set filesystem nfs

[head->category*[default*]->fsmounts*[/cm/shared*]% commit 

 

12. Boot up the rest of the nodes inside of your cluster

Tags: -

Related entries:

You cannot comment on this entry