Chapter 1. Preparing for a minor update
Keep your Red Hat OpenStack Platform (RHOSP) 17.1 environment updated with the latest packages and containers.
Use the upgrade path for the following versions:
| Old RHOSP Version | New RHOSP Version |
|---|---|
| Red Hat OpenStack Platform 17.0.z | Red Hat OpenStack Platform 17.1 latest |
| Red Hat OpenStack Platform 17.1.z | Red Hat OpenStack Platform 17.1 latest |
Minor update workflow
A minor update of your RHOSP environment involves updating the RPM packages and containers on the undercloud and overcloud host, and the service configuration, if needed. The data plane and control plane are fully available during the minor update. You must complete each of the following steps to update your RHOSP environment:
| Update step | Description |
|---|---|
| Undercloud update | Director packages are updated, containers are replaced, and the undercloud is rebooted. |
|
|
All |
| ha-image-update external | Updates container image names of Pacemaker-controlled services. There is no service disruption. This step applies to only customers that are updating their system from version 17.0.z to the latest 17.1 release. |
| Overcloud update of Controller nodes and composable nodes that contain Pacemaker services | During an Overcloud update, the Pacemaker services are stopped for each host. While the Pacemaker services are stopped, the RPMs on the host, the container configuration data, and the containers are updated. When the Pacemaker services restart, the host is added again. |
| Overcloud update of composable nodes without Pacemaker services | Networker, ObjectStorage, BlockStorage, or any other role that does not include Pacemaker services are updated one node at a time. |
| Overcloud update of Compute nodes | Multiple nodes are updated in parallel. The default value for running nodes in parallel is 25. |
| Overcloud update of Ceph nodes | Ceph nodes are updated one node at a time. |
| Ceph cluster update |
Ceph services are updated by using |
If you have a multistack infrastructure, update each overcloud stack completely, one at a time. If you have a distributed compute node (DCN) infrastructure, update the overcloud at the central location completely, and then update the overcloud at each edge site, one at a time.
Additionally, an administrator can perform the following operations during a minor update:
- Migrate your virtual machine
- Create a virtual machine network
- Run additional cloud operations
The following operations are not supported during a minor update:
- Replacing a Controller node
- Scaling in or scaling out any role
Considerations before you update your RHOSP environment
To help guide you during the update process, consider the following information:
- Red Hat recommends backing up the undercloud and overcloud control planes. For more information about backing up nodes, see Backing up and restoring the undercloud and control plane nodes.
- Familiarize yourself with the known issues that might block an update.
- Familiarize yourself with the possible update and upgrade paths before you begin your update. For more information, see Section 1.1, “Upgrade paths for long life releases”.
-
To identify your current maintenance release, run
$ cat /etc/rhosp-release. You can also run this command after updating your environment to validate the update.
Updates with a single Controller node are not supported.
Procedure
To prepare your RHOSP environment for the minor update, complete the following procedures:
1.1. Upgrade paths for long life releases Copy linkLink copied to clipboard!
Familiarize yourself with the possible update and upgrade paths before you begin an update.
You can view your current RHOSP and RHEL versions in the /etc/rhosp-release and /etc/redhat-release files.
| Current version | Target version |
|---|---|
| RHOSP 17.0.x on RHEL 9.0 | RHOSP 17.0 latest on RHEL 9.0 latest |
| RHOSP 17.1.x on RHEL 9.2 | RHOSP 17.1 latest on RHEL 9.2 latest |
| Current version | Target version |
|---|---|
| RHOSP 10 on RHEL 7.7 | RHOSP 13 latest on RHEL 7.9 latest |
| RHOSP 13 on RHEL 7.9 | RHOSP 16.1 latest on RHEL 8.2 latest |
| RHOSP 13 on RHEL 7.9 | RHOSP 16.2 latest on RHEL 8.4 latest |
| RHOSP 16 on RHEL 8.4 | RHOSP 17.1 latest on RHEL 9.0 latest |
For more information, see Framework for upgrades (16.2 to 17.1).
1.2. Known issues that might block a minor update Copy linkLink copied to clipboard!
Review the following known issues that might affect a successful update.
During an update from Red Hat OpenStack Platform (RHOSP) 17.1 GA, 17.1.1, and 17.1.2 to RHOSP 17.1.4, when you use an Open Virtual Network (OVN) back end, there is a possibility of a short network API outage during the external run of the OVN update.
If your Red Hat OpenStack Platform (RHOSP) 17.0 environment is deployed with ML2/OVN, you cannot update your environment directly from RHOSP 17.0 to 17.1.4. You must update to RHOSP 17.0.1 first. For more information, see Keeping Red Hat OpenStack Platform Updated.
If your RHOSP 17.1.3 or earlier deployment includes a filter rule in nftables or iptables with a LOG action, and the kernel command line (/proc/cmdline) has console=tty50, logging actions can cause substantial latency in packet transmission. Before updating to 17.1.4, you must apply the workaround in the Red Hat Knowledgebase solution Sometimes receiving packet(e.g. ICMP echo) has latency, around 190(ms).
If you are performing an update of your RHOSP environment to 17.1.x, the pre-update package_version validation fails because the validation cannot find a matching podman version.
Workaround: To skip the package_version validation, use the --skiplist package-version option when you run the pre-update validation:
$ validation run -i ~/overcloud-deploy/<stack>/tripleo-ansible-inventory.yaml --group pre-update --skiplist package-version
-
Replace
<stack>with the name of your stack.
During an update to RHOSP 17.1.4, when you run the pre-update validation, the undercloud-service-status validation fails. The failure occurs because the validation cannot find the undercloud-service-status service.
Workaround: Skip the undercloud-service-status validation when you run the pre-update validation:
$ validation run \
--group pre-update \
--inventory /home/stack/overcloud-deploy/<stack>/config-download/<stack>/tripleo-ansible-inventory.yaml
--skiplist undercloud-service-status
-
Replace
<stack>with the name of your stack.
1.3. Repository adjustments Copy linkLink copied to clipboard!
You must adjust which repositories you enabled based on the current phase of the Red Hat Enterprise Linux Life Cycle. You should select Enhanced Extended Update Service (EEUS) if it is available in your entitlements to minimize the need for future adjustments. If you do not have the EEUS repositories, then select the Extended Update Support (EUS) repositories. If your entitlements do not currently provide the right repositories, contact Red Hat support.
Depending on your Red Hat OpenStack Platform subscription entitlements, you have one of the following repository types:
RHEL 8 or Multi-RHEL environments:
Telecommunications Update Service (TUS)
-
rhel-8-for-x86_64-baseos-tus-rpms -
rhel-8-for-x86_64-appstream-tus-rpms -
rhel-8-for-x86_64-highavailability-tus-rpms
-
Advanced Update Stream (AUS)
-
rhel-8-for-x86_64-baseos-aus-rpms -
rhel-8-for-x86_64-appstream-aus-rpms -
rhel-8-for-x86_64-highavailability-aus-rpms
-
RHEL 9 environments:
Extended Update Support (EUS)
-
rhel-9-for-x86_64-baseos-eus-rpms -
rhel-9-for-x86_64-appstream-eus-rpms -
rhel-9-for-x86_64-highavailability-eus-rpms
-
Enhanced Extended Update Service (EEUS)
-
rhel-9-for-x86_64-baseos-e4s-rpms -
rhel-9-for-x86_64-appstream-e4s-rpms -
rhel-9-for-x86_64-highavailability-e4s-rpms
-
Advanced Update Stream (AUS)
-
rhel-9-for-x86_64-baseos-aus-rpms -
rhel-9-for-x86_64-appstream-aus-rpms -
rhel-9-for-x86_64-highavailability-aus-rpms
-
To learn more about the repositories that are included with your subscription entitlement, see Red Hat Enterprise Linux Life Cycle.
1.4. Locking the environment to a Red Hat Enterprise Linux release Copy linkLink copied to clipboard!
Red Hat OpenStack Platform (RHOSP) 17.1 is supported on Red Hat Enterprise Linux (RHEL) 9.2. Before you perform the update, lock the undercloud and overcloud repositories to the RHEL 9.2 release to avoid upgrading the operating system to a newer minor release.
Procedure
-
Log in to the undercloud host as the
stackuser. Source the
stackrcundercloud credentials file:$ source ~/stackrc-
Edit your overcloud subscription management environment file, which is the file that contains the
RhsmVarsparameter. The default name for this file is usuallyrhsm.yml. Check if your subscription management configuration includes the
rhsm_releaseparameter. If therhsm_releaseparameter is not present, add it and set it to 9.2:parameter_defaults: RhsmVars: … rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "1a85f9223e3d5e43013e3d6e8ff506fd" rhsm_method: "portal" rhsm_release: "9.2"- Save the overcloud subscription management environment file.
Create a playbook that contains a task to lock the operating system version to RHEL 9.2 on all nodes:
$ cat > ~/set_release.yaml <<'EOF' - hosts: all gather_facts: false tasks: - name: set release to 9.2 command: subscription-manager release --set=9.2 become: true EOFRun the
set_release.yamlplaybook:$ ansible-playbook -i ~/overcloud-deploy/<stack>/tripleo-ansible-inventory.yaml -f 25 ~/set_release.yaml --limit <undercloud>, <Controller>, <Compute>-
Replace
<stack>with the name of your stack. -
Use the
--limitoption to apply the content to all RHOSP nodes. Replace <undercloud>, <Controller>, <Compute> with the Ansible groups in your environment that contain those nodes. Do not run this playbook against Ceph Storage nodes because you might have a different subscription for these nodes.
-
Replace
To manually lock a node to a version, log in to the node and run the subscription-manager release command:
$ sudo subscription-manager release --set=9.2
1.5. Updating Red Hat Openstack Platform repositories Copy linkLink copied to clipboard!
Update your repositories to use Red Hat OpenStack Platform (RHOSP) 17.1.
Procedure
-
Log in to the undercloud host as the
stackuser. Source the
stackrcundercloud credentials file:$ source ~/stackrc-
Edit your overcloud subscription management environment file, which is the file that contains the
RhsmVarsparameter. The default name for this file is usuallyrhsm.yml. Check the
rhsm_reposparameter in your subscription management configuration. If therhsm_reposparameter is using the RHOSP 17.1 repositories, change the repository to the correct versions:parameter_defaults: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-e4s-rpms - rhel-9-for-x86_64-appstream-e4s-rpms - rhel-9-for-x86_64-highavailability-e4s-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms- Save the overcloud subscription management environment file.
Create a playbook that contains a task to set the repositories to RHOSP 17.1 on all nodes:
$ cat > ~/update_rhosp_repos.yaml <<'EOF' - hosts: all gather_facts: false tasks: - name: change osp repos command: subscription-manager repos --enable=openstack-17.1-for-rhel-9-x86_64-rpms become: true EOFRun the
update_rhosp_repos.yamlplaybook:$ ansible-playbook -i ~/overcloud-deploy/<stack>/tripleo-ansible-inventory.yaml -f 25 ~/update_rhosp_repos.yaml --limit <undercloud>,<Controller>,<Compute>-
Replace
<stack>with the name of your stack. -
Use the
--limitoption to apply the content to all RHOSP nodes. Replace <undercloud>, <Controller>, and <Compute> with the Ansible groups in your environment that contain those nodes. Do not run this playbook against Ceph Storage nodes because they usually use a different subscription.
-
Replace
Create a playbook that contains a task to set the repositories to RHOSP 17.1 on all ceph storage nodes:
$ cat > ~/update_ceph_repos.yaml <<'EOF' - hosts: all gather_facts: false tasks: - name: change ceph repos command: subscription-manager repos --enable=openstack-17.1-deployment-tools-for-rhel-9-x86_64-rpms become: true EOFRun the
update_ceph_repos.yamlplaybook:$ ansible-playbook -i ~/overcloud-deploy/<stack>/tripleo-ansible-inventory.yaml -f 25 ~/update_ceph_repos.yaml --limit CephStorageUse the
--limitoption to apply the content to Ceph Storage nodes.
1.6. Updating the container image preparation file Copy linkLink copied to clipboard!
The container preparation file is the file that contains the ContainerImagePrepare parameter. You use this file to define the rules for obtaining container images for the undercloud and overcloud.
Before you update your environment, check the file to ensure that you obtain the correct image versions.
Procedure
-
Edit the container preparation file. The default name for this file is usually
containers-prepare-parameter.yaml. Ensure that the
tagparameter is set to17.1for each rule set:parameter_defaults: ContainerImagePrepare: - push_destination: true set: ... tag: '17.1' tag_from_label: '{version}-{release}'NoteIf you do not want to use a specific tag for the update, such as
17.1or17.1.1, remove thetagkey-value pair and specifytag_from_labelonly. This uses the installed Red Hat OpenStack Platform version to determine the value for the tag to use as part of the update process. For more information about version tagging, see Guidelines for container image tagging in Customizing your Red Hat OpenStack Platform deployment.- Save this file.
1.7. Disabling fencing in the overcloud Copy linkLink copied to clipboard!
Before you update the overcloud, ensure that fencing is disabled.
If fencing is deployed in your environment during the Controller nodes update process, the overcloud might detect certain nodes as disabled and attempt fencing operations, which can cause unintended results.
If you have enabled fencing in the overcloud, you must temporarily disable fencing for the duration of the update.
Procedure
-
Log in to the undercloud host as the
stackuser. Source the
stackrcundercloud credentials file:$ source ~/stackrcFor each Controller node, log in to the Controller node and run the Pacemaker command to disable fencing:
$ ssh tripleo-admin@<controller_ip> "sudo pcs property set stonith-enabled=false"-
Replace
<controller_ip>with the IP address of a Controller node. You can find the IP addresses of your Controller nodes at/etc/hostsor/var/lib/mistral.
-
Replace
If you use SBD fencing, disable SBD fencing on the
pacemaker_remotenodes:# pcs property set stonith-watchdog-timeout=0 --forceNoteEnsure that you take note of the original value of the watchdog timer device interval. You must reset the watchdog timer device interval to its original value after you upgrade the control plane nodes. For more information, see Re-enabling fencing in the overcloud.
-
In the
fencing.yamlenvironment file, set theEnableFencingparameter tofalseto ensure that fencing stays disabled during the update process.
Additional Resources
1.8. Firewall rule change Copy linkLink copied to clipboard!
In Red Hat OpenStack Platform (RHOSP) 17.1.4, iptables rules were replaced with nftables rules. If your RHOSP templates include firewall rules, for example, tripleo::tripleo_firewall::firewall_rules, you must redefine them by using the ExtraFirewallRules parameter. For more information about using the ExtraFirewallRules parameter, see Adding services to the overcloud firewall in Hardening Red Hat OpenStack Platform.