이 콘텐츠는 선택한 언어로 제공되지 않습니다.
Chapter 7. Preparing for an overcloud upgrade
You must complete some initial steps to prepare for the overcloud upgrade.
7.1. Preparing for overcloud service downtime 링크 복사링크가 클립보드에 복사되었습니다!
The overcloud upgrade process disables the main control plane services at key points. You cannot use any overcloud services to create new resources when these key points are reached.
It is important to plan a maintenance window to ensure that no users can access the overcloud services during the upgrade.
For information about the duration and impact of this upgrade procedure, see Upgrade duration and impact.
7.2. Disabling fencing in the overcloud 링크 복사링크가 클립보드에 복사되었습니다!
Before you upgrade the overcloud, ensure that fencing is disabled.
When you upgrade the overcloud, you upgrade each Controller node individually to retain high availability functionality. If fencing is deployed in your environment, the overcloud might detect certain nodes as disabled and attempt fencing operations, which can cause unintended results.
If you have enabled fencing in the overcloud, you must temporarily disable fencing for the duration of the upgrade to avoid any unintended results.
When you complete the upgrade of your Red Hat OpenStack Platform environment, you must re-enable fencing in the overcloud. For more information about re-enabling fencing, see Re-enabling fencing in the overcloud.
Procedure
-
Log in to the undercloud host as the
stackuser. Source the
stackrcundercloud credentials file:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow For each Controller node, log in to the Controller node and run the Pacemaker command to disable fencing:
ssh tripleo-admin@<controller_ip> "sudo pcs property set stonith-enabled=false"
$ ssh tripleo-admin@<controller_ip> "sudo pcs property set stonith-enabled=false"Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Replace
<controller_ip>with the IP address of a Controller node. You can find the IP addresses of your Controller nodes at/etc/hostsor/var/lib/mistral.
-
Replace
If you use SBD fencing, disable SBD fencing on the
pacemaker_remotenodes:pcs property set stonith-watchdog-timeout=0
# pcs property set stonith-watchdog-timeout=0Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteEnsure that you take note of the original value of the watchdog timer device interval. You must reset the watchdog timer device interval to its original value after you upgrade the control plane nodes. For more information, see Re-enabling fencing in the overcloud.
-
In the
fencing.yamlenvironment file, set theEnableFencingparameter tofalseto ensure that fencing stays disabled during the upgrade process.
Additional Resources
7.3. Undercloud node database backup 링크 복사링크가 클립보드에 복사되었습니다!
You can use the openstack undercloud backup --db-only command to create a standalone database backup that runs on the undercloud node. You can also use that backup to recover the state of the database in the event that it becomes corrupted. For more information about backing up the undercloud database, see Creating a standalone database backup of the undercloud nodes in Red Hat OpenStack Platform 17.1 Backing up and restoring the undercloud and control plane nodes.
7.4. Updating composable services in custom roles_data files 링크 복사링크가 클립보드에 복사되었습니다!
This section contains information about new and deprecated composable services.
All nodes
The following services have been deprecated for all nodes. Remove them from all roles.
| Service | Reason |
|---|---|
|
| Deprecated services. |
|
| Deprecated service. |
|
|
Deprecated in favour of |
|
| Deprecated service. |
|
| Deprecated service. |
|
| Deprecated services. |
|
| OpenStack Networking (neutron) Load Balancing as a Service deprecated in favour of Octavia. |
|
| Deprecated services. |
|
| Deprecated service. |
|
| This service is removed. |
|
|
Deprecated in favor of |
|
| OpenDaylight is no longer supported. |
|
| Deprecated services. |
|
| The OpenStack Telemetry services are deprecated in favor of Service Telemetry Framework (STF) for metrics and monitoring. The legacy telemetry services are only available in RHOSP 17.1 to help facilitate the transition to STF and will be removed in a future version of RHOSP. |
|
| Deprecated service. |
|
| Deprecated services. |
|
| Deprecated services. |
|
| Skydive is no longer supported. |
|
| Tacker is no longer supported. |
|
| Deprecated service. |
|
| Deprecated services. |
|
| Deprecated service. |
Controller nodes
The following services are new for Controller nodes. Add them to your Controller role.
| Service | Reason |
|---|---|
|
| Service for the internal instance of the Image service (glance) API to provide location data to administrators and services that require it, such as the Block Storage service (cinder) and the Compute service (nova). |
Compute nodes
By default, 17.1 Compute nodes run the OS::TripleO::Services::NovaLibvirt service. However, if you perform the RHOSP upgrade with the Compute nodes running the OS::TripleO::Services::NovaLibvirt service, the virtual machine instances appear as shut off. To prevent this issue, all Compute nodes that are on RHEL 8.4 must run the OS::TripleO::Services::NovaLibvirtLegacy service, and the container image must be based on UBI-8.
After the RHOSP upgrade, if you want to upgrade your Compute nodes to RHEL 9.2, your Compute nodes must run the OS::TripleO::Services::NovaLibvirt service and the container image must be based on UBI-9, or your virtual machine instances appear as shut off.
For more information about upgrading the operating system on Compute nodes, see Upgrading all Compute nodes to RHEL 9.2 and Upgrading Compute nodes to a Multi-RHEL environment.
7.5. Checking custom Puppet parameters 링크 복사링크가 클립보드에 복사되었습니다!
If you use the ExtraConfig interfaces for customizations of Puppet parameters, Puppet might report duplicate declaration errors during the upgrade. This is due to changes in the interfaces provided by the puppet modules themselves.
This procedure shows how to check for any custom ExtraConfig hieradata parameters in your environment files.
If your environment uses LDAP backends, remove the following deprecated parameters from the keystone_domain_specific_ldap_backend.yaml environment file to prevent overcloud upgrade failure:
-
user_allow_create -
user_allow_update -
user_allow_delete -
group_allow_create -
group_allow_update -
group_allow_delete
For more information about removing these parameters, see the Red Hat Knowledgebase solution Overcloud upgrade to RHOSP 17.1 failed due to Keystone error when deprecated ldap related options are present in templates.
Procedure
Select an environment file and the check if it has an
ExtraConfigparameter:grep ExtraConfig ~/templates/custom-config.yaml
$ grep ExtraConfig ~/templates/custom-config.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
If the results show an
ExtraConfigparameter for any role (e.g.ControllerExtraConfig) in the chosen file, check the full parameter structure in that file. If the parameter contains any puppet Hierdata with a
SECTION/parametersyntax followed by avalue, it might have been been replaced with a parameter with an actual Puppet class. For example:parameter_defaults: ExtraConfig: neutron::config::dhcp_agent_config: 'DEFAULT/dnsmasq_local_resolv': value: 'true'parameter_defaults: ExtraConfig: neutron::config::dhcp_agent_config: 'DEFAULT/dnsmasq_local_resolv': value: 'true'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Check the director’s Puppet modules to see if the parameter now exists within a Puppet class. For example:
grep dnsmasq_local_resolv
$ grep dnsmasq_local_resolvCopy to Clipboard Copied! Toggle word wrap Toggle overflow If so, change to the new interface.
The following are examples to demonstrate the change in syntax:
Example 1:
parameter_defaults: ExtraConfig: neutron::config::dhcp_agent_config: 'DEFAULT/dnsmasq_local_resolv': value: 'true'parameter_defaults: ExtraConfig: neutron::config::dhcp_agent_config: 'DEFAULT/dnsmasq_local_resolv': value: 'true'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Changes to:
parameter_defaults: ExtraConfig: neutron::agents::dhcp::dnsmasq_local_resolv: trueparameter_defaults: ExtraConfig: neutron::agents::dhcp::dnsmasq_local_resolv: trueCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example 2:
parameter_defaults: ExtraConfig: ceilometer::config::ceilometer_config: 'oslo_messaging_rabbit/rabbit_qos_prefetch_count': value: '32'parameter_defaults: ExtraConfig: ceilometer::config::ceilometer_config: 'oslo_messaging_rabbit/rabbit_qos_prefetch_count': value: '32'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Changes to:
parameter_defaults: ExtraConfig: oslo::messaging::rabbit::rabbit_qos_prefetch_count: '32'parameter_defaults: ExtraConfig: oslo::messaging::rabbit::rabbit_qos_prefetch_count: '32'Copy to Clipboard Copied! Toggle word wrap Toggle overflow
7.6. Firewall rule change 링크 복사링크가 클립보드에 복사되었습니다!
In Red Hat OpenStack Platform (RHOSP) 17.1.4, iptables rules were replaced with nftables rules. If your RHOSP templates include firewall rules, for example, tripleo::tripleo_firewall::firewall_rules, you must redefine them by using the ExtraFirewallRules parameter. For more information about using the ExtraFirewallRules parameter, see Adding services to the overcloud firewall in Hardening Red Hat OpenStack Platform.
7.7. Final review before upgrade 링크 복사링크가 클립보드에 복사되었습니다!
Complete a final check of all preparation steps before you begin the upgrade.
7.7.1. Checking for orphaned service records 링크 복사링크가 클립보드에 복사되었습니다!
If orphaned service records exist in the database, the Compute service (nova) might not start after the upgrade process completes. Before you start the upgrade, check for the existence of orphaned service records and remove them.
Procedure
Perform a search for services:
openstack --os-compute-api-version 2.53 compute service list
openstack --os-compute-api-version 2.53 compute service listCopy to Clipboard Copied! Toggle word wrap Toggle overflow Remove any service where the
Hostcolumn points to non-existent hosts and where theUpdated Atcolumn indicates that the service was not active during the use of the last major version you are about to upgrade from:openstack --os-compute-api-version 2.53 compute service delete <service_uuid>
openstack --os-compute-api-version 2.53 compute service delete <service_uuid>Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
<service_uuid>with the UUID of the service record.
7.7.2. Upgrade command overview 링크 복사링크가 클립보드에 복사되었습니다!
The upgrade process involves different commands that you run at certain stages of the process.
This section only contains information about each command. You must run these commands in a specific order and provide options specific to your overcloud. Wait until you receive instructions to run these commands at the appropriate step.
7.7.2.1. openstack overcloud upgrade prepare 링크 복사링크가 클립보드에 복사되었습니다!
This command performs the initial preparation steps for the overcloud upgrade, which includes replacing the current overcloud plan on the undercloud with the new OpenStack Platform 17.1 overcloud plan and your updated environment files. This command functions similar to the openstack overcloud deploy command and uses many of the same options.
Before you run the openstack overcloud upgrade prepare command, you must perform the overcloud adoption. For more information about overcloud adoption, see Performing the overcloud adoption and preparation.
7.7.2.2. openstack overcloud upgrade run 링크 복사링크가 클립보드에 복사되었습니다!
This command performs the upgrade process. Director creates a set of Ansible playbooks based on the new OpenStack Platform 17.1 overcloud plan and runs the fast forward tasks on the entire overcloud. This includes running the upgrade process through each OpenStack Platform version from 16.2 to 17.1.
In addition to the standard upgrade process, this command can perform a Leapp upgrade of the operating system on overcloud nodes. Run these tasks using the --tags option.
Upgrade task tags for Leapp
system_upgrade-
Task that combines tasks from
system_upgrade_prepare,system_upgrade_run, andsystem_upgrade_reboot. system_upgrade_prepare- Tasks to prepare for the operating system upgrade with Leapp.
system_upgrade_run- Tasks to run Leapp and upgrade the operating system.
system_upgrade_reboot- Tasks to reboot a system and complete the operating system upgrade.
7.7.2.3. openstack overcloud external-upgrade run 링크 복사링크가 클립보드에 복사되었습니다!
This command performs upgrade tasks outside the standard upgrade process. Director creates a set of Ansible playbooks based on the new OpenStack Platform 17.1 overcloud plan and you run specific tasks using the --tags option.
External task tags for container management
container_image_prepare- Tasks for pulling container images to the undercloud registry and preparing the images for the overcloud to use.
7.7.3. Upgrade Parameters 링크 복사링크가 클립보드에 복사되었습니다!
You can modify the behavior of the upgrade process with upgrade parameters.
| Parameter | Description |
|---|---|
|
| Command or script snippet to run on all overcloud nodes to initialize the upgrade process. For example, a repository switch. |
|
| Common commands required by the upgrades process. This should not normally be modified by the operator and is set and unset in the major-upgrade-composable-steps.yaml and major-upgrade-converge.yaml environment files. |
|
| Additional command line options to append to the Leapp command. |
|
|
Print debugging output when running Leapp. The default value is |
|
| Skip Leapp checks by setting env variables when running Leapp in development/testing. For example, LEAPP_DEVEL_SKIP_RHSM=1. |
|
|
Use Leapp for operating system upgrade. The default value is |
|
|
Maximum (seconds) to wait for machine to reboot and respond to a test command. The default value is |
|
|
Timeout (seconds) for the OS upgrade phase via Leapp. The default value is |
|
| List of packages to install after Leapp upgrade. |
|
| List of packages to remove during Leapp upgrade. |
7.7.4. Custom files to include in your deployment 링크 복사링크가 클립보드에 복사되었습니다!
If any overcloud nodes in your deployment are dedicated Object Storage (swift) nodes, you must copy the default roles_data.yaml file and edit ObjectStorage to remove deprecated_server_resource_name: 'SwiftStorage'. Then use the --roles-file option to pass the file to the openstack overcloud upgrade prepare command.
7.7.5. New environment files to include with your deployment 링크 복사링크가 클립보드에 복사되었습니다!
In addition to your regular overcloud environment files, you must include new environment files to facilitate the upgrade to Red Hat OpenStack Platform (RHOSP) 17.1.
| File | Notes |
|---|---|
|
| This file contains the parameters specific to the upgrade. This file is necessary only for the duration of the upgrade. |
|
| The file that contains the source and preparation steps. This is the same file that you use with the undercloud upgrade. |
|
| This file contains the parameters that Ceph Storage is required to override. |
Add these files to the end of your environment file listing when you run the following commands:
-
openstack overcloud upgrade prepare -
openstack overcloud deploy
7.7.6. Environment files to remove from your deployment 링크 복사링크가 클립보드에 복사되었습니다!
Remove any environment files specific to your OpenStack Platform Red Hat OpenStack Platform 16.2:
- Red Hat OpenStack Platform 16.2 container image list
-
Red Hat OpenStack Platform 16.2 Customer Portal or Satellite
rhel-registrationscripts
Remove these files from the list of environment files you include when you run the following commands:
-
openstack overcloud upgrade prepare -
openstack overcloud deploy
7.7.7. Upgrading IPA services 링크 복사링크가 클립보드에 복사되었습니다!
If TLS everywhere is enabled in your environment, add an additional permission to the Nova Host Manager role to allow the creation of DNS zone entries.
Procedure
-
Log in to the undercloud host as the
stackuser. Source the
stackrcundercloud credentials file:source ~/stackrc
$ source ~/stackrcCopy to Clipboard Copied! Toggle word wrap Toggle overflow Check whether the
Nova Host Managementpermission is included in your environment:ipa privilege-show "Nova Host Management"
$ ipa privilege-show "Nova Host Management"Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you do not have the
Nova Host Managementpermission, add it:kinit admin ipa privilege-add-permission 'Nova Host Management' --permission 'System: Modify Realm Domains'
$ kinit admin $ ipa privilege-add-permission 'Nova Host Management' --permission 'System: Modify Realm Domains'Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an environment file called
ipa_environment.yamland include the following configuration:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Save the environment file.
7.7.8. Upgrade checklist 링크 복사링크가 클립보드에 복사되었습니다!
Use the following checklist to determine your readiness to upgrade the overcloud:
| Item | Complete |
|---|---|
| Validated a working overcloud. | Y / N |
| Performed a Relax-and-Recover (ReaR) backup of the overcloud control plane. For more information, see Red Hat OpenStack Platform 16.2 Backing up and restoring the undercloud and control plane nodes. | Y / N |
| Created a backup of the database that runs on the undercloud node. For more information, see Creating a backup of the undercloud node in Red Hat OpenStack Platform 17.1 Backing up and restoring the undercloud and control plane nodes. | Y / N |
| Updated your registration details to Red Hat OpenStack Platform 17.1 repositories and converted your environment file to use the Ansible-based method. | Y / N |
| Updated your network configuration templates. | Y / N |
| Updated any local copies of your environment files to match the content in the Red Hat OpenStack Platform 17.1 environment files. | Y / N |
| Updated your environment file list with new environment files for Red Hat OpenStack Platform 17.1. | Y / N |
| Optional: If your deployment includes dedicated Object Storage (swift) nodes:
Copied the | Y / N |
| Removed old environment files only relevant to Red Hat OpenStack Platform 16.2, such as old Red Hat registration and container image location files. | Y / N |