16.5. Expanding the cluster
After deploying an installer-provisioned OpenShift Container Platform cluster, you can use the following procedures to expand the number of worker nodes. Ensure that each prospective worker node meets the prerequisites.
Expanding the cluster using RedFish Virtual Media involves meeting minimum firmware requirements. See Firmware requirements for installing with virtual media in the Prerequisites section for additional details when expanding the cluster using RedFish Virtual Media.
16.5.1. Preparing the bare metal node Copier lienLien copié sur presse-papiers!
To expand your cluster, you must provide the node with the relevant IP address. This can be done with a static configuration, or with a DHCP (Dynamic Host Configuration protocol) server. When expanding the cluster using a DHCP server, each node must have a DHCP reservation.
Some administrators prefer to use static IP addresses so that each node’s IP address remains constant in the absence of a DHCP server. To configure static IP addresses with NMState, see "Optional: Configuring host network interfaces in the install-config.yaml file" in the "Setting up the environment for an OpenShift installation" section for additional details.
Preparing the bare metal node requires executing the following procedure from the provisioner node.
Procédure
Get the
ocbinary:curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - oc
$ curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - ocCopy to Clipboard Copied! Toggle word wrap Toggle overflow sudo cp oc /usr/local/bin
$ sudo cp oc /usr/local/binCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Power off the bare metal node by using the baseboard management controller (BMC), and ensure it is off.
Retrieve the user name and password of the bare metal node’s baseboard management controller. Then, create
base64strings from the user name and password:echo -ne "root" | base64
$ echo -ne "root" | base64Copy to Clipboard Copied! Toggle word wrap Toggle overflow echo -ne "password" | base64
$ echo -ne "password" | base64Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a configuration file for the bare metal node. Depending on whether you are using a static configuration or a DHCP server, use one of the following example
bmh.yamlfiles, replacing values in the YAML to match your environment:vim bmh.yaml
$ vim bmh.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Static configuration
bmh.yaml:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- To configure the network interface for a newly created node, specify the name of the secret that contains the network configuration. Follow the
nmstatesyntax to define the network configuration for your node. See "Optional: Configuring host network interfaces in the install-config.yaml file" for details on configuring NMState syntax. - 2 10 13 16
- Replace
<num>for the worker number of the bare metal node in thenamefields, thecredentialsNamefield, and thepreprovisioningNetworkDataNamefield. - 3
- Add the NMState YAML syntax to configure the host interfaces.
- 4
- Optional: If you have configured the network interface with
nmstate, and you want to disable an interface, setstate: upwith the IP addresses set toenabled: falseas shown:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 5 6 7 8 9
- Replace
<nic1_name>,<ip_address>,<dns_ip_address>,<next_hop_ip_address>and<next_hop_nic1_name>with appropriate values. - 11 12
- Replace
<base64_of_uid>and<base64_of_pwd>with the base64 string of the user name and password. - 14
- Replace
<nic1_mac_address>with the MAC address of the bare metal node’s first NIC. See the "BMC addressing" section for additional BMC configuration options. - 15
- Replace
<protocol>with the BMC protocol, such as IPMI, RedFish, or others. Replace<bmc_url>with the URL of the bare metal node’s baseboard management controller. - 17
- To skip certificate validation, set
disableCertificateVerificationto true. - 18 19
- Replace
<bmc_username>and<bmc_password>with the string of the BMC user name and password. - 20
- Optional: Replace
<root_device_hint>with a device path if you specify a root device hint. - 21
- Optional: If you have configured the network interface for the newly created node, provide the network configuration secret name in the
preprovisioningNetworkDataNameof the BareMetalHost CR.
DHCP configuration
bmh.yaml:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1 4 7
- Replace
<num>for the worker number of the bare metal node in thenamefields, thecredentialsNamefield, and thepreprovisioningNetworkDataNamefield. - 2 3
- Replace
<base64_of_uid>and<base64_of_pwd>with the base64 string of the user name and password. - 5
- Replace
<nic1_mac_address>with the MAC address of the bare metal node’s first NIC. See the "BMC addressing" section for additional BMC configuration options. - 6
- Replace
<protocol>with the BMC protocol, such as IPMI, RedFish, or others. Replace<bmc_url>with the URL of the bare metal node’s baseboard management controller. - 8
- To skip certificate validation, set
disableCertificateVerificationto true. - 9 10
- Replace
<bmc_username>and<bmc_password>with the string of the BMC user name and password. - 11
- Optional: Replace
<root_device_hint>with a device path if you specify a root device hint. - 12
- Optional: If you have configured the network interface for the newly created node, provide the network configuration secret name in the
preprovisioningNetworkDataNameof the BareMetalHost CR.
NoteIf the MAC address of an existing bare metal node matches the MAC address of a bare metal host that you are attempting to provision, then the Ironic installation will fail. If the host enrollment, inspection, cleaning, or other Ironic steps fail, the Bare Metal Operator retries the installation continuously. See "Diagnosing a host duplicate MAC address" for more information.
Create the bare metal node:
oc -n openshift-machine-api create -f bmh.yaml
$ oc -n openshift-machine-api create -f bmh.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
secret/openshift-worker-<num>-network-config-secret created secret/openshift-worker-<num>-bmc-secret created baremetalhost.metal3.io/openshift-worker-<num> created
secret/openshift-worker-<num>-network-config-secret created secret/openshift-worker-<num>-bmc-secret created baremetalhost.metal3.io/openshift-worker-<num> createdCopy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<num>will be the worker number.Power up and inspect the bare metal node:
oc -n openshift-machine-api get bmh openshift-worker-<num>
$ oc -n openshift-machine-api get bmh openshift-worker-<num>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<num>is the worker node number.Exemple de sortie
NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> available true
NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> available trueCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteTo allow the worker node to join the cluster, scale the
machinesetobject to the number of theBareMetalHostobjects. You can scale nodes either manually or automatically. To scale nodes automatically, use themetal3.io/autoscale-to-hostsannotation formachineset.
16.5.2. Replacing a bare-metal control plane node Copier lienLien copié sur presse-papiers!
Use the following procedure to replace an installer-provisioned OpenShift Container Platform control plane node.
Si vous réutilisez la définition de l'objet BareMetalHost à partir d'un hôte de plan de contrôle existant, ne laissez pas le champ externallyProvisioned sur true.
Les objets du plan de contrôle BareMetalHost existants peuvent avoir l'indicateur externallyProvisioned défini sur true s'ils ont été provisionnés par le programme d'installation d'OpenShift Container Platform.
Conditions préalables
-
Vous avez accès au cluster en tant qu'utilisateur ayant le rôle
cluster-admin. Vous avez effectué une sauvegarde etcd.
ImportantTake an etcd backup before performing this procedure so that you can restore your cluster if you encounter any issues. For more information about taking an etcd backup, see the Additional resources section.
Procédure
Ensure that the Bare Metal Operator is available:
oc get clusteroperator baremetal
$ oc get clusteroperator baremetalCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE baremetal 4.12.0 True False False 3d15h
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE baremetal 4.12.0 True False False 3d15hCopy to Clipboard Copied! Toggle word wrap Toggle overflow Remove the old
BareMetalHostandMachineobjects:oc delete bmh -n openshift-machine-api <host_name> oc delete machine -n openshift-machine-api <machine_name>
$ oc delete bmh -n openshift-machine-api <host_name> $ oc delete machine -n openshift-machine-api <machine_name>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<host_name>with the name of the host and<machine_name>with the name of the machine. The machine name appears under theCONSUMERfield.After you remove the
BareMetalHostandMachineobjects, then the machine controller automatically deletes theNodeobject.Créez le nouvel objet
BareMetalHostet le secret pour stocker les informations d'identification BMC :Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1 4 6
- Replace
<num>for the control plane number of the bare metal node in thenamefields and thecredentialsNamefield. - 2
- Replace
<base64_of_uid>with thebase64string of the user name. - 3
- Replace
<base64_of_pwd>with thebase64string of the password. - 5
- Replace
<protocol>with the BMC protocol, such asredfish,redfish-virtualmedia,idrac-virtualmedia, or others. Replace<bmc_ip>with the IP address of the bare metal node’s baseboard management controller. For additional BMC configuration options, see "BMC addressing" in the Additional resources section. - 7
- Replace
<NIC1_mac_address>with the MAC address of the bare metal node’s first NIC.
Une fois l'inspection terminée, l'objet
BareMetalHostest créé et disponible pour être approvisionné.View available
BareMetalHostobjects:oc get bmh -n openshift-machine-api
$ oc get bmh -n openshift-machine-apiCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
Copy to Clipboard Copied! Toggle word wrap Toggle overflow There are no
MachineSetobjects for control plane nodes, so you must create aMachineobject instead. You can copy theproviderSpecfrom another control planeMachineobject.Create a
Machineobject:Copy to Clipboard Copied! Toggle word wrap Toggle overflow To view the
BareMetalHostobjects, run the following command:oc get bmh -A
$ oc get bmh -ACopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
Copy to Clipboard Copied! Toggle word wrap Toggle overflow After the RHCOS installation, verify that the
BareMetalHostis added to the cluster:oc get nodes
$ oc get nodesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteAfter replacement of the new control plane node, the etcd pod running in the new node is in
crashloopbackstatus. See "Replacing an unhealthy etcd member" in the Additional resources section for more information.
16.5.3. Preparing to deploy with Virtual Media on the baremetal network Copier lienLien copié sur presse-papiers!
If the provisioning network is enabled and you want to expand the cluster using Virtual Media on the baremetal network, use the following procedure.
Conditions préalables
-
There is an existing cluster with a
baremetalnetwork and aprovisioningnetwork.
Procédure
Edit the
provisioningcustom resource (CR) to enable deploying with Virtual Media on thebaremetalnetwork:oc edit provisioning
oc edit provisioningCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Add
virtualMediaViaExternalNetwork: trueto theprovisioningCR.
If the image URL exists, edit the
machinesetto use the API VIP address. This step only applies to clusters installed in versions 4.9 or earlier.oc edit machineset
oc edit machinesetCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow
16.5.4. Diagnosing a duplicate MAC address when provisioning a new host in the cluster Copier lienLien copié sur presse-papiers!
If the MAC address of an existing bare-metal node in the cluster matches the MAC address of a bare-metal host you are attempting to add to the cluster, the Bare Metal Operator associates the host with the existing node. If the host enrollment, inspection, cleaning, or other Ironic steps fail, the Bare Metal Operator retries the installation continuously. A registration error is displayed for the failed bare-metal host.
You can diagnose a duplicate MAC address by examining the bare-metal hosts that are running in the openshift-machine-api namespace.
Conditions préalables
- Install an OpenShift Container Platform cluster on bare metal.
-
Install the OpenShift Container Platform CLI
oc. -
Connectez-vous en tant qu'utilisateur disposant des privilèges
cluster-admin.
Procédure
To determine whether a bare-metal host that fails provisioning has the same MAC address as an existing node, do the following:
Get the bare-metal hosts running in the
openshift-machine-apinamespace:oc get bmh -n openshift-machine-api
$ oc get bmh -n openshift-machine-apiCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To see more detailed information about the status of the failing host, run the following command replacing
<bare_metal_host_name>with the name of the host:oc get -n openshift-machine-api bmh <bare_metal_host_name> -o yaml
$ oc get -n openshift-machine-api bmh <bare_metal_host_name> -o yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Exemple de sortie
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
16.5.5. Provisioning the bare metal node Copier lienLien copié sur presse-papiers!
Provisioning the bare metal node requires executing the following procedure from the provisioner node.
Procédure
Ensure the
STATEisavailablebefore provisioning the bare metal node.oc -n openshift-machine-api get bmh openshift-worker-<num>
$ oc -n openshift-machine-api get bmh openshift-worker-<num>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<num>is the worker node number.NAME STATE ONLINE ERROR AGE openshift-worker available true 34h
NAME STATE ONLINE ERROR AGE openshift-worker available true 34hCopy to Clipboard Copied! Toggle word wrap Toggle overflow Get a count of the number of worker nodes.
oc get nodes
$ oc get nodesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow Get the compute machine set.
oc get machinesets -n openshift-machine-api
$ oc get machinesets -n openshift-machine-apiCopy to Clipboard Copied! Toggle word wrap Toggle overflow NAME DESIRED CURRENT READY AVAILABLE AGE ... openshift-worker-0.example.com 1 1 1 1 55m openshift-worker-1.example.com 1 1 1 1 55m
NAME DESIRED CURRENT READY AVAILABLE AGE ... openshift-worker-0.example.com 1 1 1 1 55m openshift-worker-1.example.com 1 1 1 1 55mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Increase the number of worker nodes by one.
oc scale --replicas=<num> machineset <machineset> -n openshift-machine-api
$ oc scale --replicas=<num> machineset <machineset> -n openshift-machine-apiCopy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<num>with the new number of worker nodes. Replace<machineset>with the name of the compute machine set from the previous step.Check the status of the bare metal node.
oc -n openshift-machine-api get bmh openshift-worker-<num>
$ oc -n openshift-machine-api get bmh openshift-worker-<num>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<num>is the worker node number. The STATE changes fromreadytoprovisioning.NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> provisioning openshift-worker-<num>-65tjz true
NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> provisioning openshift-worker-<num>-65tjz trueCopy to Clipboard Copied! Toggle word wrap Toggle overflow The
provisioningstatus remains until the OpenShift Container Platform cluster provisions the node. This can take 30 minutes or more. After the node is provisioned, the state will change toprovisioned.NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> provisioned openshift-worker-<num>-65tjz true
NAME STATE CONSUMER ONLINE ERROR openshift-worker-<num> provisioned openshift-worker-<num>-65tjz trueCopy to Clipboard Copied! Toggle word wrap Toggle overflow After provisioning completes, ensure the bare metal node is ready.
oc get nodes
$ oc get nodesCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can also check the kubelet.
ssh openshift-worker-<num>
$ ssh openshift-worker-<num>Copy to Clipboard Copied! Toggle word wrap Toggle overflow [kni@openshift-worker-<num>]$ journalctl -fu kubelet
[kni@openshift-worker-<num>]$ journalctl -fu kubeletCopy to Clipboard Copied! Toggle word wrap Toggle overflow