Kubernetes is most easily installed on a cluster that is able to access the internet. For clusters without internet access it is still possible to deploy Kubernetes, with a few additional steps.
The following document refers to Bright 9.2 on RHEL8.
1. Install the following RPM packages, and their dependencies, on the head node(s) and in all the software images from which Kubernetes is to be deployed:
Packages:
- cm-containerd
- cm-docker
- cm-etcd
- cm-kubernetes-ceph-storage-class (optional)
- cm-kubernetes-local-path-provisioner
- cm-kubernetes-permissions-manager
- cm-kubernetes121
- conntrack-tools
- nginx
- nginx-all-modules
- cm-docker-registry
Dependencies:
- container-selinux
- libnetfilter_cthelper
- libnetfilter_cttimeout
- libnetfilter_queue
- nginx-mod-http-image-filter
- nginx-mod-http-perl
- nginx-mod-http-xslt-filter
- nginx-mod-mail
- nginx-mod-stream
- socat
2. Setup a container registry on the head node using cm-container-registry-setup script:
|
# cm-container-registry-setup |
Note: for simplicity, you can choose docker-registry and deploy it on the head node and choose all default settings.
3. Start docker service to be able to save images to the registry
| # systemctl start docker.service |
4. Create a list of container images to be loaded into the docker registry
|
# sed -n ‘s/[ -]*image: *//p’ /cm/local/apps/kubernetes/var/addons/*yaml | sort -u | tee /tmp/ctr-images.txt # echo k8s.gcr.io/pause:3.2 >> /tmp/ctr-images.txt |
5. Copy the ctr-images.txt to a computer with internet connectivity and running docker.
6. Download and save the list of container images in ctr-images.txt from the computer with internet connectivity:
| # mkdir k8s-images && cd k8s-images && for image in $(cat /tmp/ctr-images.txt); do docker pull $image; docker save $image -o ${image//\//_}.tar; done |
7. Copy the saved docker images, in tar archive format, into the head node:
8. On the Head Node, import and tag the images into the local Docker registry (installed in step 3)
|
# module load docker |
9. Run the cm-kubernetes-setup with “–skip-packages” and follow the on-screen instructions to install kubernetes as required but at the end of the wizard choose Save and Exit (not save & deploy).
| # cm-kubernetes-setup –skip-packages |
10. Replace the official registries with the local one in the generated kubernetes setup config file (default is cm-kubernetes-setup.conf):
|
# registry_address=$(hostname -f):5000; sed -i “s/docker.io/$registry_address/g;s/k8s.gcr.io/$registry_address/g;s/gcr.io/$registry_address/g;s/quay.io/$registry_address/g;” cm-kubernetes-setup.conf |
11. Add “–pod-infra-container-image=<$hostname>:5000/pause:3.2” after the last line of the kubelet section:
|
node: |
12. Replace docker registry addresses in the addon yamls
| # cp -pr /cm/local/apps/kubernetes/var/addons /cm/local/apps/kubernetes/var/addons.orig # find /cm/local/apps/kubernetes/var/addons/ -type f -name ‘*.yaml’ | xargs -n 1 sed -i “s/docker.io/$registry_address/g;s/k8s.gcr.io/$registry_address/g;s/gcr.io/$registry_address/g;s/quay.io/$registry_address/g;” |
13. Install kubernetes using the modified Run cm-kubernetes-setup using the -c option, and wait for the installation to complete:
| # cm-kubernetes-setup –c cm-kubernetes-setup.conf |
Note: If all is well, Kubernetes gets deployed without issues.
14. Verify that the kubernetes setup is running properly:
|
# module load kubernetes |