Does Bright’s Kubernetes installation stick to open cloud standards?

or:
How can I verify that Bright’s Kubernetes deployment passes the official CNCF conformance tests?

The CNCF (Cloud Native Computing Foundation) is a vendor neutral foundation that encourages standards in clouds. Container technologies such as Kubernetes are regarded as important building block ecosystems, and are used by many vendors. A common risk with a vendor implementation is a deviation from standards. This can often be unintentional.

Therefore, to help vendors adhere to standards, the software standards bodies nowadays typically provide conformance test suites.

The Kubernetes deployment that Bright provides to end users is checked internally via the CNCF test suite since Bright 8.2-9. End users can also verify it for themselves with something similar to the following procedure:

Environment

• Bright Cluster Manager 8.2-9 (minimum required version)
• Centos 7
• 6 compute nodes, each one with 4 cores, 8GB memory, 50GB disk

The cm-kubernetes-setup is then run on the head node. In the ncurses screens, when asked, the administrator should allocate:

• 3 masters (1 each) to: node001, node002, and node003
• 3 etcd (1 each) to: node001, node002 and node003 (this means a full HA configuration is set up)
• 3 workers (1 each) to: node004, node005 and node006

Preparation

The CNCF software test suite used is called sonobuoy.

An aside on the word sonobuoy (pronounced “sono boy”). In the physical world, a sonobuoy is a sonar buoy device used to probe for submarines. The name fits in with the nautical theme of kubernetes (which itself in turn is the anglicised version of the ancient Greek word for captain/helmsman). The metaphor that is followed is thus that, like for the physical sonobuoy, the software sonobuoy is meant to discover issues hidden beneath the surface.

The latest sonobuoy software release and checksum file can be picked up from the web page at https://github.com/heptio/sonobuoy/releases:

[root@head ~]# wget https://github.com/heptio/sonobuoy/releases/download/v0.14.2/sonobuoy_0.14.2_linux_amd64.tar.gz 
...
2019-05-08 17:27:52 (362 KB/s) - ‘sonobuoy_0.14.2_linux_amd64.tar.gz’ saved [11037258/11037258]
2019-05-08 17:27:53 (7.04 MB/s) - ‘sonobuoy_0.14.2_checksums.txt’ saved [306/306]
...

The checksum can be tested with:

[root@head ~]# sha256sum sonobuoy_0.14.2_linux_amd64.tar.gz
30ed9604750b61ea99ea11238172701c61e3149758e46c1c5d7f5e13ad779764  sonobuoy_0.14.2_linux_amd64.tar.gz
[root@head ~]# grep linux sonobuoy_0.14.2_checksums.txt
30ed9604750b61ea99ea11238172701c61e3149758e46c1c5d7f5e13ad779764  sonobuoy_0.14.2_linux_amd64.tar.gz

Unpacking and running the conformance test suite:

The tar.gz file can be untarred and extracted with:

[root@head ~]# tar tvzf sonobuoy_0.14.2_linux_amd64.tar.gz
-rw-rw-r-- travis/travis 11357 2019-04-29 21:24 LICENSE
-rwxrwxr-x travis/travis 34590848 2019-04-29 21:44 sonobuoy
[root@head ~]# tar xvzf sonobuoy_0.14.2_linux_amd64.tar.gz
LICENSE
sonobuoy

The conformance test can be run with:

[root@head ~]# ./sonobuoy run --wait
Running plugins: e2e, systemd-logs
INFO[0000] created object            name=heptio-sonobuoy namespace= resource=namespaces
INFO[0000] created object            name=sonobuoy-serviceaccount namespace=heptio-sonobuoy resource=serviceaccounts
INFO[0000] created object            name=sonobuoy-serviceaccount-heptio-sonobuoy namespace= resource=clusterrolebindings
...

Completion can take maybe 1 or 2 hours.

Checking the test results:
Once a run has been completed, the results can be checked for failures with:
[root@head ~]# ./sonobuoy e2e $(./sonobuoy retrieve)
failed tests: 0
```

0 failed tests means the test suite was passed successfully, and that the Bright Kubernetes deployment indeed conforms to the CNCF standards.

Cleanup of the sonobuoy files:

The files created during the test suite run can be deleted with:

[root@head ~]# ./sonobuoy delete --wait
INFO[0000] deleted                                       kind=namespace namespace=heptio-sonobuoy
INFO[0000] deleted                                       kind=clusterrolebindings
INFO[0000] deleted                                       kind=clusterroles
```