Chapter 8. Adding more RHEL compute machines to an OpenShift Container Platform cluster

1. On each host, register with RHSM:

   # subscription-manager register --username=<user_name> --password=<password>

2. Pull the latest subscription data from RHSM:

   # subscription-manager refresh

3. List the available subscriptions:

   # subscription-manager list --available --matches '*OpenShift*'

4. In the output for the previous command, find the pool ID for an OpenShift Container Platform subscription and attach it:

   # subscription-manager attach --pool=<pool_id>

5. Disable all yum repositories:

   1. Disable all the enabled RHSM repositories:

      # subscription-manager repos --disable="*"

   2. List the remaining yum repositories and note their names under repo id, if any:

      # yum repolist

   3. Use yum-config-manager to disable the remaining yum repositories:

      # yum-config-manager --disable <repo_id>

      Alternatively, disable all repositories:

       yum-config-manager --disable \*

      Note that this might take a few minutes if you have a large number of available repositories

6. Enable only the repositories required by OpenShift Container Platform 4.2:

   # subscription-manager repos \
       --enable="rhel-7-server-rpms" \
       --enable="rhel-7-server-extras-rpms" \
       --enable="rhel-7-server-ose-4.2-rpms"

7. Stop and disable firewalld on the host:

   # systemctl disable --now firewalld.service

   Note

   You must not enable firewalld later. If you do, you cannot access OpenShift Container Platform logs on the worker.

Procedure

1. Open the Ansible inventory file at /<path>/inventory/hosts that defines your compute machine hosts and required variables.
2. Rename the [new_workers] section of the file to [workers].

3. Add a [new_workers] section to the file and define the fully-qualified domain names for each new host. The file resembles the following example:

   [all:vars]
   ansible_user=root
   #ansible_become=True
   
   openshift_kubeconfig_path="~/.kube/config"
   
   [workers]
   mycluster-worker-0.example.com
   mycluster-worker-1.example.com
   
   [new_workers]
   mycluster-worker-2.example.com
   mycluster-worker-3.example.com

   In this example, the mycluster-worker-0.example.com and mycluster-worker-1.example.com machines are in the cluster and you add the mycluster-worker-2.example.com and mycluster-worker-3.example.com machines.

4. Run the scaleup playbook:

   $ cd /usr/share/ansible/openshift-ansible
   $ ansible-playbook -i /<path>/inventory/hosts playbooks/scaleup.yml 1

   1

   For <path>, specify the path to the Ansible inventory file that you created.

Procedure

1. Confirm that the cluster recognizes the machines:

   $ oc get nodes
   
   NAME      STATUS    ROLES   AGE  VERSION
   master-0  Ready     master  63m  v1.14.6+c4799753c
   master-1  Ready     master  63m  v1.14.6+c4799753c
   master-2  Ready     master  64m  v1.14.6+c4799753c
   worker-0  NotReady  worker  76s  v1.14.6+c4799753c
   worker-1  NotReady  worker  70s  v1.14.6+c4799753c

   The output lists all of the machines that you created.

2. Review the pending certificate signing requests (CSRs) and ensure that the you see a client and server request with Pending or Approved status for each machine that you added to the cluster:

   $ oc get csr
   
   NAME        AGE     REQUESTOR                                                                   CONDITION
   csr-8b2br   15m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending 1
   csr-8vnps   15m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Pending
   csr-bfd72   5m26s   system:node:ip-10-0-50-126.us-east-2.compute.internal                       Pending 2
   csr-c57lv   5m26s   system:node:ip-10-0-95-157.us-east-2.compute.internal                       Pending
   ...

   1

   A client request CSR.

   2

   A server request CSR.

   In this example, two machines are joining the cluster. You might see more approved CSRs in the list.

3. If the CSRs were not approved, after all of the pending CSRs for the machines you added are in Pending status, approve the CSRs for your cluster machines:

   Note

   Because the CSRs rotate automatically, approve your CSRs within an hour of adding the machines to the cluster. If you do not approve them within an hour, the certificates will rotate, and more than two certificates will be present for each node. You must approve all of these certificates. After you approve the initial CSRs, the subsequent node client CSRs are automatically approved by the cluster kube-controller-manager. You must implement a method of automatically approving the kubelet serving certificate requests.

   • To approve them individually, run the following command for each valid CSR:

     $ oc adm certificate approve <csr_name> 1

     1

     <csr_name> is the name of a CSR from the list of current CSRs.

   • To approve all pending CSRs, run the following command:

     $ oc get csr -o go-template='{{range .items}}{{if not .status}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}' | xargs oc adm certificate approve