Chapter 9. Updating a cluster using the CLI
You can perform minor version and patch updates on an OpenShift Container Platform cluster by using the OpenShift CLI (oc
).
9.1. Prerequisites Copy linkLink copied to clipboard!
-
Have access to the cluster as a user with
admin
privileges. See Using RBAC to define and apply permissions. - Have a recent etcd backup in case your update fails and you must restore your cluster to a previous state.
- Support for RHEL7 workers is removed in OpenShift Container Platform 4.12. You must replace RHEL7 workers with RHEL8 or RHCOS workers before upgrading to OpenShift Container Platform 4.12. Red Hat does not support in-place RHEL7 to RHEL8 updates for RHEL workers; those hosts must be replaced with a clean operating system install.
- Ensure all Operators previously installed through Operator Lifecycle Manager (OLM) are updated to their latest version in their latest channel. Updating the Operators ensures they have a valid update path when the default OperatorHub catalogs switch from the current minor version to the next during a cluster update. See Updating installed Operators for more information.
- Ensure that all machine config pools (MCPs) are running and not paused. Nodes associated with a paused MCP are skipped during the update process. You can pause the MCPs if you are performing a canary rollout update strategy.
- If your cluster uses manually maintained credentials, update the cloud provider resources for the new release. For more information, including how to determine if this is a requirement for your cluster, see Preparing to update a cluster with manually maintained credentials.
-
Ensure that you address all
Upgradeable=False
conditions so the cluster allows an update to the next minor version. An alert displays at the top of the Cluster Settings page when you have one or more cluster Operators that cannot be upgraded. You can still update to the next available patch update for the minor release you are currently on. - Review the list of APIs that were removed in Kubernetes 1.25, migrate any affected components to use the new API version, and provide the administrator acknowledgment. For more information, see Preparing to update to OpenShift Container Platform 4.12.
-
If you run an Operator or you have configured any application with the pod disruption budget, you might experience an interruption during the upgrade process. If
minAvailable
is set to 1 inPodDisruptionBudget
, the nodes are drained to apply pending machine configs which might block the eviction process. If several nodes are rebooted, all the pods might run on only one node, and thePodDisruptionBudget
field can prevent the node drain.
- When an update is failing to complete, the Cluster Version Operator (CVO) reports the status of any blocking components while attempting to reconcile the update. Rolling your cluster back to a previous version is not supported. If your update is failing to complete, contact Red Hat support.
-
Using the
unsupportedConfigOverrides
section to modify the configuration of an Operator is unsupported and might block cluster updates. You must remove this setting before you can update your cluster.
9.2. Pausing a MachineHealthCheck resource Copy linkLink copied to clipboard!
During the upgrade process, nodes in the cluster might become temporarily unavailable. In the case of worker nodes, the machine health check might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, pause all the MachineHealthCheck
resources before updating the cluster.
Prerequisites
-
Install the OpenShift CLI (
oc
).
Procedure
To list all the available
MachineHealthCheck
resources that you want to pause, run the following command:oc get machinehealthcheck -n openshift-machine-api
$ oc get machinehealthcheck -n openshift-machine-api
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To pause the machine health checks, add the
cluster.x-k8s.io/paused=""
annotation to theMachineHealthCheck
resource. Run the following command:oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused=""
$ oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused=""
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The annotated
MachineHealthCheck
resource resembles the following YAML file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantResume the machine health checks after updating the cluster. To resume the check, remove the pause annotation from the
MachineHealthCheck
resource by running the following command:oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused-
$ oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused-
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.3. About updating single node OpenShift Container Platform Copy linkLink copied to clipboard!
You can update, or upgrade, a single-node OpenShift Container Platform cluster by using either the console or CLI.
However, note the following limitations:
-
The prerequisite to pause the
MachineHealthCheck
resources is not required because there is no other node to perform the health check. - Restoring a single-node OpenShift Container Platform cluster using an etcd backup is not officially supported. However, it is good practice to perform the etcd backup in case your upgrade fails. If your control plane is healthy, you might be able to restore your cluster to a previous state by using the backup.
Updating a single-node OpenShift Container Platform cluster requires downtime and can include an automatic reboot. The amount of downtime depends on the update payload, as described in the following scenarios:
- If the update payload contains an operating system update, which requires a reboot, the downtime is significant and impacts cluster management and user workloads.
- If the update contains machine configuration changes that do not require a reboot, the downtime is less, and the impact on the cluster management and user workloads is lessened. In this case, the node draining step is skipped with single-node OpenShift Container Platform because there is no other node in the cluster to reschedule the workloads to.
- If the update payload does not contain an operating system update or machine configuration changes, a short API outage occurs and resolves quickly.
There are conditions, such as bugs in an updated package, that can cause the single node to not restart after a reboot. In this case, the update does not rollback automatically.
9.4. Updating a cluster by using the CLI Copy linkLink copied to clipboard!
You can use the OpenShift CLI (oc
) to review and request cluster updates.
You can find information about available OpenShift Container Platform advisories and updates in the errata section of the Customer Portal.
Prerequisites
-
Install the OpenShift CLI (
oc
) that matches the version for your updated version. -
Log in to the cluster as user with
cluster-admin
privileges. -
Pause all
MachineHealthCheck
resources.
Procedure
View the available updates and note the version number of the update that you want to apply:
oc adm upgrade
$ oc adm upgrade
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Note- If there are no available updates, updates that are supported but not recommended might still be available. See Updating along a conditional update path for more information.
-
For details and information on how to perform an
EUS-to-EUS
channel update, please refer to the Preparing to perform an EUS-to-EUS upgrade page, listed in the Additional resources section.
Based on your organization requirements, set the appropriate update channel. For example, you can set your channel to
stable-4.13
orfast-4.13
. For more information about channels, refer to Understanding update channels and releases listed in the Additional resources section.oc adm upgrade channel <channel>
$ oc adm upgrade channel <channel>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow For example, to set the channel to
stable-4.12
:oc adm upgrade channel stable-4.12
$ oc adm upgrade channel stable-4.12
Copy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantFor production clusters, you must subscribe to a
stable-*
,eus-*
, orfast-*
channel.NoteWhen you are ready to move to the next minor version, choose the channel that corresponds to that minor version. The sooner the update channel is declared, the more effectively the cluster can recommend update paths to your target version. The cluster might take some time to evaluate all the possible updates that are available and offer the best update recommendations to choose from. Update recommendations can change over time, as they are based on what update options are available at the time.
If you cannot see an update path to your target minor version, keep updating your cluster to the latest patch release for your current version until the next minor version is available in the path.
Apply an update:
To update to the latest version:
oc adm upgrade --to-latest=true
$ oc adm upgrade --to-latest=true
1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow To update to a specific version:
oc adm upgrade --to=<version>
$ oc adm upgrade --to=<version>
1 Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Review the status of the Cluster Version Operator:
oc adm upgrade
$ oc adm upgrade
Copy to Clipboard Copied! Toggle word wrap Toggle overflow After the update completes, you can confirm that the cluster version has updated to the new version:
oc get clusterversion
$ oc get clusterversion
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you are updating your cluster to the next minor version, such as version X.y to X.(y+1), it is recommended to confirm that your nodes are updated before deploying workloads that rely on a new feature:
oc get nodes
$ oc get nodes
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.5. Updating along a conditional upgrade path Copy linkLink copied to clipboard!
You can update along a recommended conditional upgrade path using the web console or the OpenShift CLI (oc
). When a conditional update is not recommended for your cluster, you can update along a conditional upgrade path using the OpenShift CLI (oc
) 4.10 or later.
Procedure
To view the description of the update when it is not recommended because a risk might apply, run the following command:
oc adm upgrade --include-not-recommended
$ oc adm upgrade --include-not-recommended
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the cluster administrator evaluates the potential known risks and decides it is acceptable for the current cluster, then the administrator can waive the safety guards and proceed the update by running the following command:
oc adm upgrade --allow-not-recommended --to <version> <.>
$ oc adm upgrade --allow-not-recommended --to <version> <.>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow <.>
<version>
is the supported but not recommended update version that you obtained from the output of the previous command.
9.6. Changing the update server by using the CLI Copy linkLink copied to clipboard!
Changing the update server is optional. If you have an OpenShift Update Service (OSUS) installed and configured locally, you must set the URL for the server as the upstream
to use the local server during updates. The default value for upstream
is https://api.openshift.com/api/upgrades_info/v1/graph
.
Procedure
Change the
upstream
parameter value in the cluster version:oc patch clusterversion/version --patch '{"spec":{"upstream":"<update-server-url>"}}' --type=merge
$ oc patch clusterversion/version --patch '{"spec":{"upstream":"<update-server-url>"}}' --type=merge
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
<update-server-url>
variable specifies the URL for the update server.Example output
clusterversion.config.openshift.io/version patched
clusterversion.config.openshift.io/version patched
Copy to Clipboard Copied! Toggle word wrap Toggle overflow