Chapter 7. Adding RHEL compute machines to an OpenShift Container Platform cluster

Procedure

1. Ensure that the kubeconfig file for the cluster and the installation program that you used to install the cluster are on the machine. One way to accomplish this is to use the same machine that you used to install the cluster.
2. Configure the machine to access all of the RHEL hosts that you plan to use as compute machines. You can use any method that your company allows, including a bastion with an SSH proxy or a VPN.

3. Configure a user on the machine that you run the playbook on that has SSH access to all of the RHEL hosts.

   Important

   If you use SSH key-based authentication, you must manage the key with an SSH agent.

4. If you have not already done so, register the machine with RHSM and attach a pool with an OpenShift subscription to it:

   1. Register the machine with RHSM:

      # subscription-manager register --username=<user_name> --password=<password>

   2. Pull the latest subscription data from RHSM:

      # subscription-manager refresh

   3. List the available subscriptions:

      # subscription-manager list --available --matches '*OpenShift*'

   4. In the output for the previous command, find the pool ID for an OpenShift Container Platform subscription and attach it:

      # subscription-manager attach --pool=<pool_id>

5. Enable the repositories required by OpenShift Container Platform 4.1:

   # subscription-manager repos \
       --enable="rhel-7-server-rpms" \
       --enable="rhel-7-server-extras-rpms" \
       --enable="rhel-7-server-ansible-2.7-rpms" \
       --enable="rhel-7-server-ose-4.1-rpms"

6. Install the required packages, including Openshift-Ansible:

   # yum install openshift-ansible openshift-clients jq

   The openshift-ansible package provides installation program utilities and pulls in other packages that you require to add a RHEL compute node to your cluster, such as Ansible, playbooks, and related configuration files. The openshift-clients provides the oc CLI, and the jq package improves the display of JSON output on your command line.

1. On each host, register with RHSM:

   # subscription-manager register --username=<user_name> --password=<password>

2. Pull the latest subscription data from RHSM:

   # subscription-manager refresh

3. List the available subscriptions:

   # subscription-manager list --available --matches '*OpenShift*'

4. In the output for the previous command, find the pool ID for an OpenShift Container Platform subscription and attach it:

   # subscription-manager attach --pool=<pool_id>

5. Disable all yum repositories:

   1. Disable all the enabled RHSM repositories:

      # subscription-manager repos --disable="*"

   2. List the remaining yum repositories and note their names under repo id, if any:

      # yum repolist

   3. Use yum-config-manager to disable the remaining yum repositories:

      # yum-config-manager --disable <repo_id>

      Alternatively, disable all repositories:

       yum-config-manager --disable \*

      Note that this might take a few minutes if you have a large number of available repositories

6. Enable only the repositories required by OpenShift Container Platform 4.1:

   # subscription-manager repos \
       --enable="rhel-7-server-rpms" \
       --enable="rhel-7-server-extras-rpms" \
       --enable="rhel-7-server-ose-4.1-rpms"

7. Stop and disable firewalld on the host:

   # systemctl disable --now firewalld.service

   Note

   You must not enable firewalld later. If you do, you cannot access OpenShift Container Platform logs on the worker.

1. Extract the pull secret for your cluster:

   $ oc -n openshift-config get -o jsonpath='{.data.\.dockerconfigjson}' secret pull-secret | base64 -d | jq .

2. Save the pull secret in a file that is named pull-secret.txt.

3. Create an Ansible inventory file that is named /<path>/inventory/hosts that defines your compute machine hosts and required variables:

   [all:vars]
   ansible_user=root 1
   #ansible_become=True 2
   
   openshift_kubeconfig_path="~/.kube/config" 3
   openshift_pull_secret_path="~/pull-secret.txt" 4
   
   [new_workers] 5
   mycluster-worker-0.example.com
   mycluster-worker-1.example.com

   1

   Specify the user name that runs the Ansible tasks on the remote compute machines.

   2

   If you do not specify root for the ansible_user, you must set ansible_become to True and assign the user sudo permissions.

   3

   Specify the path to the kubeconfig file for your cluster.

   4

   Specify the path to the file that contains the pull secret for the image registry for your cluster.

   5

   List each RHEL machine to add to your cluster. You must provide the fully-qualified domain name for each host. This name is the host name that the cluster uses to access the machine, so set the correct public or private name to access the machine.

4. Run the playbook:

   $ cd /usr/share/ansible/openshift-ansible
   $ ansible-playbook -i /<path>/inventory/hosts playbooks/scaleup.yml 1

   1

   For <path>, specify the path to the Ansible inventory file that you created.

Procedure

1. Confirm that the cluster recognizes the machines:

   $ oc get nodes
   
   NAME      STATUS    ROLES   AGE  VERSION
   master-0  Ready     master  63m  v1.13.4+b626c2fe1
   master-1  Ready     master  63m  v1.13.4+b626c2fe1
   master-2  Ready     master  64m  v1.13.4+b626c2fe1
   worker-0  NotReady  worker  76s  v1.13.4+b626c2fe1
   worker-1  NotReady  worker  70s  v1.13.4+b626c2fe1

   The output lists all of the machines that you created.

2. Review the pending certificate signing requests (CSRs) and ensure that the you see a client and server request with Pending or Approved status for each machine that you added to the cluster:

   $ oc get csr
   
   NAME        AGE     REQUESTOR                                                                   CONDITION
   csr-8b2br   15m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending 1
   csr-8vnps   15m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
   csr-bfd72   5m26s   system:node:ip-10-0-50-126.us-east-2.compute.internal                       Pending 2
   csr-c57lv   5m26s   system:node:ip-10-0-95-157.us-east-2.compute.internal                       Pending
   ...

   1

   A client request CSR.

   2

   A server request CSR.

   In this example, two machines are joining the cluster. You might see more approved CSRs in the list.

3. If the CSRs were not approved, after all of the pending CSRs for the machines you added are in Pending status, approve the CSRs for your cluster machines:

   Note

   Because the CSRs rotate automatically, approve your CSRs within an hour of adding the machines to the cluster. If you do not approve them within an hour, the certificates will rotate, and more than two certificates will be present for each node. You must approve all of these certificates. After you approve the initial CSRs, the subsequent node client CSRs are automatically approved by the cluster kube-controller-manager. You must implement a method of automatically approving the kubelet serving certificate requests.

   • To approve them individually, run the following command for each valid CSR:

     $ oc adm certificate approve <csr_name> 1

     1

     <csr_name> is the name of a CSR from the list of current CSRs.

   • If all the CSRs are valid, approve them all by running the following command:

     $ oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs oc adm certificate approve