Chapter 12. Managing control plane machines

Procedure

1. In the web portal or console for your cloud provider, take the following actions:

   1. Locate and delete the appropriate load balancer component:

      • For AWS, delete the external load balancer. The API DNS entry in the private zone already points to the internal load balancer, which uses an identical configuration, so you do not need to modify the internal load balancer.
   2. Delete the api.$clustername.$yourdomain DNS entry in the public zone.

2. Remove the external load balancers by deleting the following lines in the control plane machine set custom resource:

   providerSpec:
     value:
       loadBalancers:
       - name: lk4pj-ext 1
         type: network 2
       - name: lk4pj-int
         type: network

   1 2

   Delete this line.

Procedure

1. Edit the following line under the providerSpec field:

   providerSpec:
     value:
       ...
       instanceType: <compatible_aws_instance_type> 1

   1

   Specify a larger AWS instance type with the same base as the previous selection. For example, you can change m6i.xlarge to m6i.2xlarge or m6i.4xlarge.

2. Save your changes.

Procedure

1. In the web portal or console for your cloud provider, take the following actions:

   1. Locate and delete the appropriate load balancer component:

      • For Azure, delete the api-internal rule for the load balancer.
   2. Delete the api.$clustername.$yourdomain DNS entry in the public zone.

2. Remove the external load balancers by deleting the following lines in the control plane machine set custom resource:

   providerSpec:
     value:
       loadBalancers:
       - name: lk4pj-ext 1
         type: network 2
       - name: lk4pj-int
         type: network

   1 2

   Delete this line.

Procedure

1. Display all of the available OpenShift Container Platform images by running one of the following commands:

   • North America:

     $  az vm image list --all --offer rh-ocp-worker --publisher redhat -o table

     Example output

     Offer          Publisher       Sku                 Urn                                                             Version
     -------------  --------------  ------------------  --------------------------------------------------------------  --------------
     rh-ocp-worker  RedHat          rh-ocp-worker       RedHat:rh-ocp-worker:rh-ocpworker:4.8.2021122100               4.8.2021122100
     rh-ocp-worker  RedHat          rh-ocp-worker-gen1  RedHat:rh-ocp-worker:rh-ocp-worker-gen1:4.8.2021122100         4.8.2021122100

   • EMEA:

     $  az vm image list --all --offer rh-ocp-worker --publisher redhat-limited -o table

     Example output

     Offer          Publisher       Sku                 Urn                                                             Version
     -------------  --------------  ------------------  --------------------------------------------------------------  --------------
     rh-ocp-worker  redhat-limited  rh-ocp-worker       redhat-limited:rh-ocp-worker:rh-ocp-worker:4.8.2021122100       4.8.2021122100
     rh-ocp-worker  redhat-limited  rh-ocp-worker-gen1  redhat-limited:rh-ocp-worker:rh-ocp-worker-gen1:4.8.2021122100  4.8.2021122100

   Note

   Regardless of the version of OpenShift Container Platform that you install, the correct version of the Azure Marketplace image to use is 4.8. If required, your VMs are automatically upgraded as part of the installation process.

2. Inspect the image for your offer by running one of the following commands:

   • North America:

     $ az vm image show --urn redhat:rh-ocp-worker:rh-ocp-worker:<version>

   • EMEA:

     $ az vm image show --urn redhat-limited:rh-ocp-worker:rh-ocp-worker:<version>

3. Review the terms of the offer by running one of the following commands:

   • North America:

     $ az vm image terms show --urn redhat:rh-ocp-worker:rh-ocp-worker:<version>

   • EMEA:

     $ az vm image terms show --urn redhat-limited:rh-ocp-worker:rh-ocp-worker:<version>

4. Accept the terms of the offering by running one of the following commands:

   • North America:

     $ az vm image terms accept --urn redhat:rh-ocp-worker:rh-ocp-worker:<version>

   • EMEA:

     $ az vm image terms accept --urn redhat-limited:rh-ocp-worker:rh-ocp-worker:<version>

5. Record the image details of your offer, specifically the values for publisher, offer, sku, and version.

6. Add the following parameters to the providerSpec section of your machine set YAML file using the image details for your offer:

   Sample providerSpec image values for Azure Marketplace machines

   providerSpec:
     value:
       image:
         offer: rh-ocp-worker
         publisher: redhat
         resourceID: ""
         sku: rh-ocp-worker
         type: MarketplaceWithPlan
         version: 4.8.2021122100

Procedure

1. In a text editor, open the YAML file for an existing machine set or create a new one.

2. Edit the following line under the providerSpec field:

   apiVersion: machine.openshift.io/v1
   kind: ControlPlaneMachineSet
   ...
   spec:
     template:
       spec:
         providerSpec:
           value:
             disks:
               type: pd-ssd 1

   1

   Control plane nodes must use the pd-ssd disk type.

Procedure

1. In a text editor, open the YAML file for an existing machine set or create a new one.

2. Edit the following section under the providerSpec field:

   apiVersion: machine.openshift.io/v1
   kind: ControlPlaneMachineSet
   ...
   spec:
     template:
       spec:
         providerSpec:
           value:
             confidentialCompute: Enabled 1
             onHostMaintenance: Terminate 2
             machineType: n2d-standard-8 3
   ...

   1

   Specify whether Confidential VM is enabled. Valid values are Disabled or Enabled.

   2

   Specify the behavior of the VM during a host maintenance event, such as a hardware or software update. For a machine that uses Confidential VM, this value must be set to Terminate, which stops the VM. Confidential VM does not support live VM migration.

   3

   Specify a machine type that supports Confidential VM. Confidential VM supports the N2D and C2D series of machine types.

Procedure

1. In a text editor, open the YAML file for an existing machine set or create a new one.

2. Edit the following section under the providerSpec field:

   apiVersion: machine.openshift.io/v1
   kind: ControlPlaneMachineSet
   ...
   spec:
     template:
       spec:
         providerSpec:
           value:
             shieldedInstanceConfig: 1
               integrityMonitoring: Enabled 2
               secureBoot: Disabled 3
               virtualizedTrustedPlatformModule: Enabled 4
   ...

   1

   In this section, specify any Shielded VM options that you want.

   2

   Specify whether UEFI Secure Boot is enabled. Valid values are Disabled or Enabled.

   3

   Specify whether integrity monitoring is enabled. Valid values are Disabled or Enabled.

   Note

   When integrity monitoring is enabled, you must not disable virtual trusted platform module (vTPM).

   4

   Specify whether vTPM is enabled. Valid values are Disabled or Enabled.

Procedure

1. To allow a specific service account to use your KMS key and to grant the service account the correct IAM role, run the following command with your KMS key name, key ring name, and location:

   $ gcloud kms keys add-iam-policy-binding <key_name> \
     --keyring <key_ring_name> \
     --location <key_ring_location> \
     --member "serviceAccount:service-<project_number>@compute-system.iam.gserviceaccount.com” \
     --role roles/cloudkms.cryptoKeyEncrypterDecrypter

2. Configure the encryption key under the providerSpec field in your machine set YAML file. For example:

   apiVersion: machine.openshift.io/v1
   kind: ControlPlaneMachineSet
   ...
   spec:
     template:
       spec:
         providerSpec:
           value:
             disks:
             - type:
               encryptionKey:
                 kmsKey:
                   name: machine-encryption-key 1
                   keyRing: openshift-encrpytion-ring 2
                   location: global 3
                   projectID: openshift-gcp-project 4
                 kmsKeyServiceAccount: openshift-service-account@openshift-gcp-project.iam.gserviceaccount.com 5

   1

   The name of the customer-managed encryption key that is used for the disk encryption.

   2

   The name of the KMS key ring that the KMS key belongs to.

   3

   The GCP location in which the KMS key ring exists.

   4

   Optional: The ID of the project in which the KMS key ring exists. If a project ID is not set, the machine set projectID in which the machine set was created is used.

   5

   Optional: The service account that is used for the encryption request for the given KMS key. If a service account is not set, the Compute Engine default service account is used.

   When a new machine is created by using the updated providerSpec object configuration, the disk encryption key is encrypted with the KMS key.

1. A control plane machine is slated for deletion.
2. The control plane machine enters the Deleting phase.

3. To satisfy the preDrain lifecycle hook, the etcd Operator takes the following actions:

   1. The etcd Operator waits until a fourth control plane machine is added to the cluster as an etcd member. This new etcd member has a state of Running but not ready until it receives the full database update from the etcd leader.
   2. When the new etcd member receives the full database update, the etcd Operator promotes the new etcd member to a voting member and removes the old etcd member from the cluster.

   After this transition is complete, it is safe for the old etcd pod and its data to be removed, so the preDrain lifecycle hook is removed.

4. The control plane machine status condition Drainable is set to True.

5. The machine controller attempts to drain the node that is backed by the control plane machine.

   • If draining fails, Drained is set to False and the machine controller attempts to drain the node again.
   • If draining succeeds, Drained is set to True.
6. The control plane machine status condition Drained is set to True.
7. If no other Operators have added a preTerminate lifecycle hook, the control plane machine status condition Terminable is set to True.
8. The machine controller removes the instance from the infrastructure provider.
9. The machine controller deletes the Node object.