Categories

ID #1423

How can I run a cluster-on-demand in Azure from within a Docker container in Bright?

How do I run a cluster-on-demand in Azure from a docker container running from Bright?

 

For convenience, running docker from within Bright is described here.

 

However, it is also possible to launch a cluster-on-demand from a docker container, from a standalone machine that has docker-engine installed on it, by following the method described here.

The method when launching from a Bright cluster is as follows:

  • install Docker in Bright with cm-docker-setup or the Bright View Docker setup wizard.

 

  • login to a node that is to run Docker, and pull the COD image into the node:
      docker pull brightcomputing/cod:8.1

    If you want it permanently available, pull it into the software image and then have the nodes pick it up via a reboot or imageupdate

 

  • Find the appropriate image ID, by grepping for it with 'brightcomputing/cod'. For example:
      [root@node002 ~]# docker images | head -1 ; docker images | grep brightcomputing/cod
      REPOSITORY                     TAG       IMAGE ID            CREATED             SIZE
      brightcomputing/cod            8.1       aeddb120e78f        5 hours ago         593 MB
  • run that image in a Docker container. This is done using the image ID from the preceding output to specify the image that is to be run, and go into the container. For example:
        [root@node002 ~]# docker run -it aeddb120e78f  /bin/bash
        [root@743d346dca20 /]#


  • In that container:
    generate an ssh key with ssh-keygen. This is used as credentials for the Azure cluster. 

  save the keys outside the container, in case you leave the container, but still need to access the Azure cluster from elsewhere.

 

For example:
  cat /root/.ssh/id_rsa.pub
  cat /root/.ssh/id_rsa

 

 and copy paste the keys outside the container.

 

 

  • Launch an Azure cluster with appropriate values for the cm-cod-azure command. This command is built up with several options:

 

      • Azure credentials, location, cluster name, node type
        Option to cm-cod-azure
        example of what a value can look like
          --azure-client-id               afe13723-c80a-68e2-9cd0-e95a657106a
          --azure-client-secret  aiVohwi7OhJ6igim=
          --azure-tenant-id  2c89c8dd-6b8b-5393-60f4-d876cbe188f
          --azure-subscription-id  5e519b1e-aff9-e839-bc3a-c5d87e9d0d5
         --azure-location  westeurope
          cluster_name  freddyazurecluster
         --node-type  Standard_D1_v2
         --head-node-type  Standard_D1_v2
        The first four --azure-* options in the preceding table require the actual Azure credentials that the cluster adminstrator obtains from Microsoft.

      • Distinguished name values
        The verify-license utility shows the distinguished name values for the options --license-{organization/unit/locality/state/country}.
        For example:

        [root@mycluster etc]# verify-license /cm/local/apps/cmd/etc/cluster.pem /cm/local/apps/cmd/etc/cluster.key info
        ========= Certificate Information ========
        information about organization, unit etc
      • product key value
        It is normally not possible to use the product key of the host cluster itself. So, normally a fresh product key must be used with the cm-cod-azure command. 

An example of a cm-cod-azure command that can be run is then:
cm-cod-azure cluster create                                              \
    --azure-location 'westeurope'                                        \
    --azure-client-id 'afe13723-c80a-68e2-9cd0-e95a657106a'              \
    --azure-client-secret 'aiVohwi7OhJ6igim='                            \
    --azure-tenant-id '2c89c8dd-6b8b-5393-60f4-d876cbe188f'              \
    --azure-subscription-id '5e519b1e-aff9-e839-bc3a-c5d87e9d0d5'        \
    --version '8.1' --wlm 'dont-configure'                               \
    --head-node-type 'Standard_D1_v2' --head-node-root-volume-size '50'  \
    --nodes '3' --node-type 'Standard_D1_v2'                             \
    --ssh-pub-key-path '/root/.ssh/id_rsa.pub'                           \
    --license-organization '' --license-unit '' --license-locality ''    \
    --license-state '' --license-country 'US'                            \
    --license-product-key '12334-324234-3413413-32-324' 'freddyazurecluster'



If all is well, then something similar to the following is displayed:

13:22:50:     INFO: Credentials are valid and have read/write authorizations
13:23:23:     INFO: Cluster Create
13:23:24:     INFO: ---------------------------------------------------------------------
13:23:24:     INFO:    Cluster:   freddyazurecluster
13:23:24:     INFO: ---------------------------------------------------------------------
13:23:24:     INFO:      Image:   bcm-cod-image-8.1-4.vhd
13:23:24:     INFO: Image date:   2018-02-27 09:01:27+00:00
13:23:24:     INFO: Head nodes:   1 (Standard_D1_V2)
13:23:24:     INFO:      Nodes:   3 (Standard_D1_v2)
13:23:24:     INFO:     Region:   westeurope
13:23:24:     INFO:   Key path:   /root/.ssh/id_rsa.pub
13:23:24:     INFO: ---------------------------------------------------------------------
Press ENTER to continue and create the cluster.
Press ctrl+c (or type 'a') to abort. Type 'i' for more info.

13:23:56:     INFO: ## Progress: 2
13:23:56:     INFO: #### stage: Creating resource group freddyazurecluster_cod_resource_group
13:23:58:     INFO: Creating storage account freddyazureclusterstorageaccount
13:24:22:     INFO: ## Progress: 5
13:24:22:     INFO: #### stage: Building deployment template
13:24:22:     INFO: generating cloud-init script
13:24:22:     INFO: ## Progress: 7
13:24:22:     INFO: #### stage: Copying head node image
13:24:22:     INFO: Copying https://brightimages.blob.core.windows.net/images/bcm-cod-image-8.1-4.vhd to freddyazureclusterstorageaccount/images/freddyazurecluster-head-node-os-disk.vhd
13:24:22:     INFO: ## Progress: 12.0
13:24:22:     INFO: #### stage: Server side copy
13:25:01:     INFO: ## Progress: 12.2
13:25:01:     INFO: #### stage: Server side copy
13:25:05:     INFO: ## Progress: 12.4
13:25:05:     INFO: #### stage: Server side copy
13:25:09:     INFO: ## Progress: 12.6
...
13:35:32:     INFO: #### stage: Server side copy
13:35:35:     INFO: ## Progress: 61.77
13:35:35:     INFO: #### stage: Server side copy
13:35:38:     INFO: ## Progress: 61.97
13:35:38:     INFO: #### stage: Server side copy
13:35:40:     INFO: Elapsed: 11:18 min
13:35:40:     INFO: ## Progress: 85
13:35:40:     INFO: #### stage: Creating and deploying Head node
13:39:53:     INFO: Waiting for CMDaemon to come up
...
13:41:34:     INFO: Waiting for CMDaemon to come up
13:41:34:     INFO: CMDaemon is not up yet, waiting 10 seconds...
13:41:34:     INFO: ## Progress: 100
13:41:34:     INFO: #### stage: Deployment finished successfully.
13:41:34:     INFO: ----------------------------------------------------------------
13:41:34:     INFO:      Cluster:   freddyazurecluster
13:41:34:     INFO: ----------------------------------------------------------------
13:41:34:     INFO:        Image:   bcm-cod-image-8.1-4.vhd
13:41:34:     INFO:   Image date:   2018-02-27 09:01:27+00:00
13:41:34:     INFO:   Head nodes:   1 (Standard_D1_V2)
13:41:34:     INFO:        Nodes:   3 (Standard_D1_v2)
13:41:34:     INFO:       Region:   westeurope
13:41:34:     INFO:     Key path:   /root/.ssh/id_rsa.pub
13:41:34:     INFO: Head node ID:   3e6d2b24-2c7b-40cb-a594-fb3e66f31319
13:41:34:     INFO:    Public IP:   13.93.90.5
13:41:34:     INFO: ----------------------------------------------------------------
[root@743d346dca20 /]#



 

  • to list the clusters running with those credentials:


[root@743d346dca20 /]# cm-cod-azure cluster list                                    \
    --azure-location 'westeurope'                                                   \

    --azure-client-id 'afe13723-c80a-68e2-9cd0-e95a657106a'                         \
    --azure-client-secret 'aiVohwi7OhJ6igim='                                       \
    --azure-tenant-id '2c89c8dd-6b8b-5393-60f4-d876cbe188f'                         \
    --azure-subscription-id '5e519b1e-aff9-e839-bc3a-c5d87e9d0d5'


+--------------------+------------------------------+------------+------------+-...
| Cluster Name       | Head node name               | Public IP  | Location   | ...
+--------------------+------------------------------+------------+------------+-...
| freddyazurecluster | freddyazurecluster-head_node | 13.98.90.5 | westeurope | ...
+--------------------+------------------------------+------------+------------+-...

 

  • the cluster can be removed with:
    [root@743d346dca20 /]# cm-cod-azure cluster delete                                 \
        --azure-location 'westeurope'                                                  \
        --azure-client-id 'afe13723-c80a-68e2-9cd0-e95a657106a'                        \
        --azure-client-secret 'aiVohwi7OhJ6igim='                                      \
        --azure-tenant-id '2c89c8dd-6b8b-5393-60f4-d876cbe188f'                        \
        --azure-subscription-id '5e519b1e-aff9-e839-bc3a-c5d87e9d0d5'                  \
        'freddyazurecluster'



Tags: -

Related entries:

You cannot comment on this entry