Chapter 5. Scaling a user-provisioned cluster with the Bare Metal Operator

Procedure

1. Create a Provisioning CR.

   1. Save the following YAML in the provisioning.yaml file:

      apiVersion: metal3.io/v1alpha1
      kind: Provisioning
      metadata:
        name: provisioning-configuration
      spec:
        provisioningNetwork: "Disabled"
        watchAllNamespaces: false

      Note

      OpenShift Container Platform 4.14 does not support enabling a provisioning network when you scale a user-provisioned cluster by using the Bare Metal Operator.

2. Create the Provisioning CR by running the following command:

   $ oc create -f provisioning.yaml

   Example output

   provisioning.metal3.io/provisioning-configuration created

Procedure

1. Create the Secret CR and the BareMetalHost CR.

   1. Save the following YAML in the bmh.yaml file:

      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: worker1-bmc
        namespace: openshift-machine-api
      type: Opaque
      data:
        username: <base64_of_uid>
        password: <base64_of_pwd>
      ---
      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: worker1
        namespace: openshift-machine-api
      spec:
        bmc:
          address: <protocol>://<bmc_url> 1
          credentialsName: "worker1-bmc"
        bootMACAddress: <nic1_mac_address>
        externallyProvisioned: false 2
        customDeploy:
          method: install_coreos
        online: true
        userData:
          name: worker-user-data-managed
          namespace: openshift-machine-api

      1

      You can only use bare-metal host drivers that support virtual media networking booting, for example redfish-virtualmedia and idrac-virtualmedia.

      2

      You must set the spec.externallyProvisioned specification in the BareMetalHost custom resource to false. The default value is false.

2. Create the bare-metal host object by running the following command:

   $ oc create -f bmh.yaml

   Example output

   secret/worker1-bmc created
   baremetalhost.metal3.io/worker1 created

3. Approve all certificate signing requests (CSRs).

   1. Verify that the provisioning state of the host is provisioned by running the following command:

      $ oc get bmh -A

      Example output

      NAMESPACE               NAME          STATE                    CONSUMER   ONLINE   ERROR   AGE
      openshift-machine-api   controller1   externally provisioned              true             5m25s
      openshift-machine-api   worker1       provisioned                         true             4m45s

   2. Get the list of pending CSRs by running the following command:

      $ oc get csr

      Example output

      NAME        AGE   SIGNERNAME                                    REQUESTOR                                         REQUESTEDDURATION CONDITION
      csr-gfm9f   33s   kubernetes.io/kube-apiserver-client-kubelet   system:serviceaccount:openshift-machine-config-o
      perator:node-bootstrapper   <none>              Pending

   3. Approve the CSR by running the following command:

      $ oc adm certificate approve <csr_name>

      Example output

      certificatesigningrequest.certificates.k8s.io/<csr_name> approved

Procedure

1. Create the Secret CR and the BareMetalHost CR.

   1. Save the following YAML in the controller.yaml file:

      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: controller1-bmc
        namespace: openshift-machine-api
      type: Opaque
      data:
        username: <base64_of_uid>
        password: <base64_of_pwd>
      ---
      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: controller1
        namespace: openshift-machine-api
      spec:
        bmc:
          address: <protocol>://<bmc_url> 1
          credentialsName: "controller1-bmc"
        bootMACAddress: <nic1_mac_address>
        customDeploy:
          method: install_coreos
        externallyProvisioned: true 2
        online: true
        userData:
          name: controller-user-data-managed
          namespace: openshift-machine-api

      1

      You can only use bare-metal host drivers that support virtual media networking booting, for example redfish-virtualmedia and idrac-virtualmedia.

      2

      You must set the value to true to prevent the BMO from re-provisioning the bare-metal controller host.

2. Create the bare-metal host object by running the following command:

   $ oc create -f controller.yaml

   Example output

   secret/controller1-bmc created
   baremetalhost.metal3.io/controller1 created

Procedure

1. Cordon and drain the host by running the following command:

   $ oc adm drain app1 --force --ignore-daemonsets=true

   Example output

   node/app1 cordoned
   WARNING: ignoring DaemonSet-managed Pods: openshift-cluster-node-tuning-operator/tuned-tvthg, openshift-dns/dns-
   default-9q6rz, openshift-dns/node-resolver-zvt42, openshift-image-registry/node-ca-mzxth, openshift-ingress-cana
   ry/ingress-canary-qq5lf, openshift-machine-config-operator/machine-config-daemon-v79dm, openshift-monitoring/nod
   e-exporter-2vn59, openshift-multus/multus-additional-cni-plugins-wssvj, openshift-multus/multus-fn8tg, openshift
   -multus/network-metrics-daemon-5qv55, openshift-network-diagnostics/network-check-target-jqxn2, openshift-ovn-ku
   bernetes/ovnkube-node-rsvqg
   evicting pod openshift-operator-lifecycle-manager/collect-profiles-27766965-258vp
   evicting pod openshift-operator-lifecycle-manager/collect-profiles-27766950-kg5mk
   evicting pod openshift-operator-lifecycle-manager/collect-profiles-27766935-stf4s
   pod/collect-profiles-27766965-258vp evicted
   pod/collect-profiles-27766950-kg5mk evicted
   pod/collect-profiles-27766935-stf4s evicted
   node/app1 drained

2. Delete the customDeploy specification from the BareMetalHost CR.

   1. Edit the BareMetalHost CR for the host by running the following command:

      $ oc edit bmh -n openshift-machine-api <host_name>

   2. Delete the lines spec.customDeploy and spec.customDeploy.method:

      ...
        customDeploy:
          method: install_coreos

   3. Verify that the provisioning state of the host changes to deprovisioning by running the following command:

      $ oc get bmh -A

      Example output

      NAMESPACE               NAME          STATE                    CONSUMER   ONLINE   ERROR   AGE
      openshift-machine-api   controller1   externally provisioned              true             58m
      openshift-machine-api   worker1       deprovisioning                      true             57m

3. Delete the node by running the following command:

   $ oc delete node <node_name>