Chapter 8. Overcloud adoption and preparation

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. Verify that the following files that were exported during the undercloud upgrade contain the expected configuration for the overcloud upgrade. You can find the following files in the ~/overcloud-deploy/$(<stack>) directory:

   • tripleo-<stack>-passwords.yaml
   • tripleo-<stack>-network-data.yaml
   • tripleo-<stack>-virtual-ips.yaml
   • <stack>-network-environment.yaml

   • tripleo-<stack>-baremetal-deployment.yaml

     • Replace <stack> with the name of your stack.
     • Check that the mappings between the baremetal node UUIDs, hostnames, and IP addresses in the tripleo-<stack>-baremetal-deployment.yaml file match your current overcloud. Correct the mappings if required.
     • If the tripleo-<stack>-passwords.yaml file does not exist, review the resolution in the Red Hat Knowledgebase article Create a missing overcloud password file.
   Warning

   You must address the issues in Jinja2 comments, and then remove all Jinja2 comment blocks, from each nic-config template that is created by automated conversion tools such as convert_v1_net_data.py or convert_heat_nic_config_to_ansible_j2.py during the undercloud upgrade. If you do not remove these comment blocks, scaled-out nodes fail to provision, or you can lose control plane networking if network_config_update is set to true.

   {{# NOTE! Custom parameter {} was hard-coded in '
   'the converted template. To parameterize use the '
   '{{role.name}}ExtraGroupVars THT interface and update the '
   'template to use an ansible var. +#}}

4. On the main stack, copy the passwords.yaml file to the ~/overcloud-deploy/$(<stack>) directory. Repeat this step on each stack in your environment:

   $ cp ~/overcloud-deploy/<stack>/tripleo-<stack>-passwords.yaml ~/overcloud-deploy/<stack>/<stack>-passwords.yaml

5. On the main stack, copy the network-data.yaml file to the stack user’s home directory and deploy the networks. Repeat this step on each stack in your environment:

   $ cp ~/overcloud-deploy/<stack>/tripleo-<stack>-network-data.yaml ~/
   $ openstack overcloud network provision --debug \
   --output /home/stack/templates/generated-networks-deployed.yaml tripleo-<stack>-network-data.yaml

   For more information, see Provisioning and deploying your overcloud in Installing and managing Red Hat OpenStack Platform with director.

6. If your environment has pre-provisioned nodes, complete the following steps:

   1. Make a backup copy of the tripleo-<stack>-virtual-ips.yaml file in the stack user’s home directory:

      $ cp ~/overcloud-deploy/<stack>/tripleo-<stack>-virtual-ips.yaml ~/
      ~/overcloud-deploy/<stack>/tripleo-<stack>-virtual-ips.yaml.orig

   2. Include the following configuration at the end of the tripleo-<stack>-virtual-ips.yaml file:

      dns_name: overcloud
        ip_address: <ctrlplane_virtual_ip> 

      1

      
        name: control_virtual_ip 

      2

      
        network: ctlplane
        subnet: ctlplane-subnet

      1

      Replace <ctrlplane_virtual_ip> with the VIP of the control plane.

      2

      The default name of the VIP.

7. On the main stack, copy the virtual-ips.yaml file to the stack user’s home directory and provision the network VIPs. Repeat this step on each stack in your environment:

   $ cp ~/overcloud-deploy/<stack>/tripleo-<stack>-virtual-ips.yaml ~/
   $ openstack overcloud network vip provision --debug \
   --stack <stack> --output \
   /home/stack/templates/generated-vip-deployed.yaml tripleo-<stack>-virtual-ips.yaml

8. On the main stack, copy the network-environment.yaml file to the stack user’s home directory. Repeat this step on each stack in your environment:

   $ cp ~/overcloud-deploy/<stack>/<stack>-network-environment.yaml ~/templates/<stack>-network-environment.yaml

   Note

   Ensure that the NetworkConfigTemplate parameters point to the generated .j2 files and that you delete the resource_registry mappings in the overcloud-network-environment.yaml file. For more information, see Manually converting NIC templates to Jinja2 Ansible format in Customizing your Red Hat OpenStack Platform deployment.

9. On the main stack, copy the baremetal-deployment.yaml file to the stack user’s home directory and provision the overcloud nodes. Repeat this step on each stack in your environment:

   $ cp ~/overcloud-deploy/<stack>/tripleo-<stack>-baremetal-deployment.yaml ~/
   $ openstack overcloud node provision --debug --stack <stack> \
   --output /home/stack/templates/baremetal-deployment.yaml \
   tripleo-<stack>-baremetal-deployment.yaml

   Note

   This is the final step of the overcloud adoption. If your overcloud adoption takes longer than 10 minutes to complete, contact Red Hat Support.

10. Complete the following steps to prepare the containers:

    1. Back up the containers-prepare-parameter.yaml file that you used for the undercloud upgrade:

       $ cp containers-prepare-parameter.yaml \
       containers-prepare-parameter.yaml.orig

    2. Define the following environment variables before you run the script to update the containers-prepare-parameter.yaml file:

       • NAMESPACE: The namespace for the UBI9 images. For example, NAMESPACE='"namespace":"example.redhat.com:5002",'
       • EL8_NAMESPACE: The namespace for the UBI8 images.
       • NEUTRON_DRIVER: The driver to use and determine which OpenStack Networking (neutron) container to use. Set to the type of containers you used to deploy the original stack. For example, set to NEUTRON_DRIVER='"neutron_driver":"ovn",' to use OVN-based containers.

       • EL8_TAGS: The tags of the UBI8 images, for example, EL8_TAGS='"tag":"17.1",'.

         • Replace "17.1", with the tag that you use in your content view.

       • EL9_TAGS: The tags of the UBI9 images, for example, EL9_TAGS='"tag":"17.1",'.

         • Replace "17.1", with the tag that you use in your content view.

           For more information about the tag parameter, see Container image preparation parameters in Customizing your Red Hat OpenStack Platform deployment.

       • CONTROL_PLANE_ROLES: The list of control plane roles using the --role option, for example, --role ControllerOpenstack, --role Database, --role Messaging, --role Networker, --role CephStorage. To view the list of control plane roles in your environment, run the following command:

         $ export STACK=<stack> \
         $ sudo awk '/tripleo_role_name/ {print "--role " $2}' \
         /var/lib/mistral/${STACK}/tripleo-ansible-inventory.yaml \
         | grep -vi compute

         • Replace <stack> with the name of your stack.

       • COMPUTE_ROLES: The list of Compute roles using the --role option, for example, --Compute-1. To view the list of Compute roles in your environment, run the following command:

         $ sudo awk '/tripleo_role_name/ {print "--role " $2}' \
         /var/lib/mistral/${STACK}/tripleo-ansible-inventory.yaml \
         | grep -i compute

       • CEPH_OVERRIDE: If you deployed Red Hat Ceph Storage, specify the Red Hat Ceph Storage 5 container images. For example:

         CEPH_OVERRIDE='"ceph_image":"rhceph-5-rhel8","ceph_tag":"<latest>",'

         • Replace <latest> with the latest ceph_tag version, for example, 5-499.

           The following is an example of the containers-prepare-parameter.yaml file configuration:

           NAMESPACE='"namespace":"registry.redhat.io/rhosp-rhel9",'
           EL8_NAMESPACE='"namespace":"registry.redhat.io/rhosp-rhel8",'
           NEUTRON_DRIVER='"neutron_driver":"ovn",'
           EL8_TAGS='"tag":"17.1",'
           EL9_TAGS='"tag":"17.1",'
           CONTROL_PLANE_ROLES="--role Controller"
           COMPUTE_ROLES="--role Compute1 --role Compute2"
           CEPH_TAGS='"ceph_tag":"5",'

    3. Run the following script to to update the containers-prepare-parameter.yaml file:

       Warning

       If you deployed Red Hat Ceph Storage, ensure that the CEPH_OVERRIDE environment variable is set to the correct values before executing the following command. Failure to do so results in issues when upgrading Red Hat Ceph Storage.

       $ python3 /usr/share/openstack-tripleo-heat-templates/tools/multi-rhel-container-image-prepare.py \
            ${COMPUTE_ROLES} \
            ${CONTROL_PLANE_ROLES} \
            --enable-multi-rhel \
            --excludes collectd \
            --excludes nova-libvirt \
            --minor-override "{${EL8_TAGS}${EL8_NAMESPACE}${CEPH_OVERRIDE}${NEUTRON_DRIVER}\"no_tag\":\"not_used\"}" \
            --major-override "{${EL9_TAGS}${NAMESPACE}${CEPH_OVERRIDE}${NEUTRON_DRIVER}\"no_tag\":\"not_used\"}" \
            --output-env-file \
           /home/stack/containers-prepare-parameter.yaml

       The multi-rhel-container-image-prepare.py script supports the following parameters:

       --output-env-file

       Writes the environment file that contains the default ContainerImagePrepare value.

       --local-push-destination

       Triggers an upload to a local registry.

       --enable-registry-login

       Enables the flag that allows the system to attempt to log in to a remote registry prior to pulling the containers. Use this flag when --local-push-destination is not used and the target systems have network connectivity to remote registries. Do not use this flag for an overcloud that might not have network connectivity to a remote registry.

       --enable-multi-rhel

       Enables multi-rhel.

       --excludes

       Lists the services to exclude.

       --major-override

       Lists the override parameters for a major release.

       --minor-override

       Lists the override parameters for a minor release.

       --role

       The list of roles.

       --role-file

       The role_data.yaml file.

    4. If you deployed Red Hat Ceph Storage, open the containers-prepare-parameter.yaml file to confirm that the Red Hat Ceph Storage 5 container images are specified and that there are no references to Red Hat Ceph Storage 6 container images.

11. If you have a director-deployed Red Hat Ceph Storage deployment, create a file called ceph_params.yaml and include the following content:

    parameter_defaults:
      CephSpecFqdn: true
      CephConfigPath: "/etc/ceph"
      CephAnsibleRepo: "rhceph-5-tools-for-rhel-8-x86_64-rpms"
      DeployedCeph: true
      GrafanaPlugins: []

    Important

    Do not remove the ceph_params.yaml file after the RHOSP upgrade is complete. This file must be present in director-deployed Red Hat Ceph Storage environments. Additionally, any time you run openstack overcloud deploy, you must include the ceph_params.yaml file, for example, -e ceph_params.yaml.

    Note

    If your Red Hat Ceph Storage deployment includes short names, you must set the CephSpecFqdn parameter to false. If set to true, the inventory generates with both the short names and domain names, causing the Red Hat Ceph Storage upgrade to fail.

12. Create an environment file called upgrades-environment.yaml in your templates directory and include the following content:

    parameter_defaults:
      ExtraConfig:
        nova::workarounds::disable_compute_service_check_for_ffu: true
      DnsServers: ["<dns_servers>"]
      DockerInsecureRegistryAddress: [<undercloud_FQDN>]
      UpgradeInitCommand: |
        sudo subscription-manager repos --disable=*
          if $( grep -q  9.2  /etc/os-release )
          then
            sudo subscription-manager repos --enable=rhel-9-for-x86_64-baseos-e4s-rpms --enable=rhel-9-for-x86_64-appstream-e4s-rpms --enable=rhel-9-for-x86_64-highavailability-e4s-rpms --enable=openstack-17.1-for-rhel-9-x86_64-rpms --enable=fast-datapath-for-rhel-9-x86_64-rpms
            sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-9-x86_64-rpms
            sudo subscription-manager release --set=9.2
          else
            sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-aus-rpms --enable=rhel-8-for-x86_64-appstream-aus-rpms --enable=rhel-8-for-x86_64-highavailability-aus-rpms --enable=openstack-17.1-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms
            sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-8-x86_64-rpms
            sudo subscription-manager release --set=8.4
          fi
    
          if $(sudo podman ps | grep -q ceph )
          then
            sudo dnf -y install cephadm
          fi

    • Replace <dns_servers> with a comma-separated list of your DNS server IP addresses, for example, ["10.0.0.36", "10.0.0.37"].

    • Replace <undercloud_FQDN> with the fully qualified domain name (FQDN) of the undercloud host, for example, ["undercloud-0.ctlplane.redhat.local:8787"].

      For more information about the upgrade parameters that you can configure in the environment file, see Upgrade parameters.

13. On the undercloud, create a file called overcloud_upgrade_prepare.sh in your templates directory. You must create this file for each stack in your environment. This file includes the original content of your overcloud deploy file and the environment files that are relevant to your environment. For example:

    #!/bin/bash
    openstack overcloud upgrade prepare --yes \
      --timeout 460 \
      --templates /usr/share/openstack-tripleo-heat-templates \
      --ntp-server 192.168.24.1 \
      --stack <stack> \
      -r /home/stack/roles_data.yaml \
      -e /home/stack/templates/internal.yaml \
      -e /home/stack/templates/network/network-environment.yaml \
      -e /home/stack/templates/inject-trust-anchor.yaml \
      -e /home/stack/templates/hostnames.yml \
      -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
      -e /home/stack/templates/nodes_data.yaml \
      -e /home/stack/templates/debug.yaml \
      -e /home/stack/templates/firstboot.yaml \
      -e /home/stack/templates/upgrades-environment.yaml \
      -e /home/stack/overcloud-params.yaml \
      -e /home/stack/templates/overcloud-network-environment.yaml \
      -e /usr/share/openstack-tripleo-heat-templates/environments/nova-hw-machine-type-upgrade.yaml \
      -e /home/stack/skip_rhel_release.yaml \
      -e ~/containers-prepare-parameter.yaml \
      -e /home/stack/templates/baremetal-deployment.yaml \
      -e /home/stack/templates/generated-networks-deployed.yaml \
      -e /home/stack/templates/generated-vip-deployed.yaml

    1. In the original network-environment.yaml file (/home/stack/templates/network/network-environment.yaml), remove all the resource_registry resources that point to OS::TripleO::*::Net::SoftwareConfig.

    2. In the overcloud_upgrade_prepare.sh file, include the following options relevant to your environment:

       • The environment file (upgrades-environment.yaml) with the upgrade-specific parameters (-e).
       • The environment file (containers-prepare-parameter.yaml) with your new container image locations (-e). In most cases, this is the same environment file that the undercloud uses.
       • The environment file (skip_rhel_release.yaml) with the release parameters (-e).
       • Any custom configuration environment files (-e) relevant to your deployment.
       • If applicable, your custom roles (roles_data) file by using --roles-file.
       • For Ceph deployments, the environment file (ceph_params.yaml) with the Ceph parameters (-e).
       • If applicable, the environment file (ipa-environment.yaml) with your IPA service (-e).
       • If you are using composable networks, the (network_data) file by using --network-file.
       • The files that were generated during overcloud adoption (networks-deployed.yaml, vip-deployed.yaml,baremetal-deployment.yaml) (-e). These files must be placed last in the overcloud upgrade prepare script.

       • If you use OVS, add the OVS environment files in the following order:

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-sriov.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs-dpdk.yaml

       • If you use OVN, add the OVN environment files in the following order:

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-dpdk.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-sriov.yaml

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-ha.yaml

           Note

           Do not include the network-isolation.yaml file in your overcloud deploy file or the overcloud_upgrade_prepare.sh file. Network isolation is defined in the network_data.yaml file.

       • If you use a custom stack name, pass the name with the --stack option.

         Note

         You must include the nova-hw-machine-type-upgrade.yaml file in your templates until all of your RHEL 8 Compute nodes are upgraded to RHEL 9 in the environment. If this file is excluded, an error appears in the nova_compute.log in the /var/log/containers/nova directory. After you upgrade all of your RHEL 8 Compute nodes to RHEL 9, you can remove this file from your configuration and update the stack.

    3. In the director-deployed Red Hat Ceph Storage use case, if you enabled the Shared File Systems service (manila) with CephFS through NFS on the deployment that you are upgrading, you must specify an additional environment file at the end of the overcloud_upgrade_prepare.sh script file. You must add the environment file at the end of the script because it overrides another environment file that is specified earlier in the script:

       -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/manila-cephfsganesha-config.yaml

    4. In the external Red Hat Ceph Storage use case, if you enabled the Shared File Systems service (manila) with CephFS through NFS on the deployment that you are upgrading, you must check that the associated environment file in the overcloud_upgrade_prepare.sh script points to the tripleo-based ceph-nfs role. If present, remove the following environment file:

       -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/manila-cephfsganesha-config.yaml

       And add the following environment file:

       -e /usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml

14. Run the upgrade preparation script for each stack in your environment:

    $ source stackrc
    $ chmod 755 /home/stack/overcloud_upgrade_prepare.sh
    $ sh /home/stack/overcloud_upgrade_prepare.sh

15. Wait until the upgrade preparation completes.

16. Download the container images:

    $ openstack overcloud external-upgrade run --stack <stack> --tags container_image_prepare

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

3. Verify that the following files that were exported during the undercloud upgrade contain the expected configuration for the overcloud upgrade. You can find the following files in the ~/overcloud-deploy/$(<stack>) directory:

   • tripleo-<stack>-network-data.yaml
   • tripleo-<stack>-virtual-ips.yaml

   • tripleo-<stack>-baremetal-deployment.yaml

     • Replace <stack> with the name of your stack.
     • Check that the mappings between the baremetal node UUIDs, hostnames, and IP addresses in the tripleo-<stack>-baremetal-deployment.yaml file match your current overcloud. Correct the mappings if required.
   Warning

   You must address the issues in Jinja2 comments, and then remove all Jinja2 comment blocks, from each nic-config template that is created by automated conversion tools such as convert_v1_net_data.py or convert_heat_nic_config_to_ansible_j2.py during the undercloud upgrade. If you do not remove these comment blocks, scaled-out nodes fail to provision, or you can lose control plane networking if network_config_update is set to true.

   {{# NOTE! Custom parameter {} was hard-coded in '
   'the converted template. To parameterize use the '
   '{{role.name}}ExtraGroupVars THT interface and update the '
   'template to use an ansible var. +#}}

4. On the main stack, copy the network-data.yaml file to the stack user’s home directory and deploy the networks. Repeat this step on each cell stack in your environment:

   Note

   The network-data.yaml file is available only on the overcloud stack. You must copy the file from the overcloud stack to all the other cell stacks.

   $ cp ~/overcloud-deploy/<overcloud-stack>/tripleo-<overcloud-stack>-network-data.yaml ~/
   $ openstack overcloud network provision --debug \
   --output /home/stack/templates/generated-<overcloud-stack>-networks-deployed.yaml tripleo-<overcloud-stack>-network-data.yaml

   For more information, see Provisioning and deploying your overcloud in Installing and managing Red Hat OpenStack Platform with director.

5. If your environment has pre-provisioned nodes, complete the following steps in each stack in your environment, starting with the overcloud stack:

   1. Make a backup copy of the tripleo-<overcloud-stack>-virtual-ips.yaml file in the stack user’s home directory:

      $ cp ~/overcloud-deploy/<overcloud-stack>/tripleo-<overcloud-stack>-virtual-ips.yaml ~/
      ~/overcloud-deploy/<overcloud-stack>/tripleo-<overcloud-stack>-virtual-ips.yaml.orig

   2. Include the following configuration at the end of the tripleo-<overcloud-stack>-virtual-ips.yaml file:

      dns_name: overcloud
        ip_address: <ctrlplane_virtual_ip> 

      1

      
        name: control_virtual_ip 

      2

      
        network: ctlplane
        subnet: ctlplane-subnet

      1

      Replace <ctrlplane_virtual_ip> with the VIP of the control plane.

      2

      The default name of the VIP.

6. On the overcloud stack, copy the virtual-ips.yaml file to the stack user’s home directory and provision the network VIPs. Repeat this step on each cell stack in your environment:

   $ cp ~/overcloud-deploy/<overcloud-stack>/tripleo-<overcloud-stack>-virtual-ips.yaml ~/
   $ openstack overcloud network vip provision --debug \
   --stack <overcloud-stack> --output \
   /home/stack/templates/generated-<overcloud-stack>-vip-deployed.yaml tripleo-<overcloud-stack>-virtual-ips.yaml

7. On the main stack, copy the baremetal-deployment.yaml file to the stack user’s home directory and provision the overcloud nodes. Repeat this step on each cell stack in your environment:

   $ cp ~/overcloud-deploy/<overcloud-stack>/tripleo-<overcloud-stack>-baremetal-deployment.yaml ~/
   $ openstack overcloud node provision --debug --stack <overcloud-stack> \
   --output /home/stack/templates/baremetal-<overcloud-stack>-deployment.yaml \
   tripleo-<overcloud-stack>-baremetal-deployment.yaml

   Note

   This is the final step of the overcloud adoption. If your overcloud adoption takes longer than 10 minutes to complete, contact Red Hat Support.

8. Complete the following steps to prepare the containers:

   1. Back up the containers-prepare-parameter.yaml file that you used for the undercloud upgrade:

      $ cp containers-prepare-parameter.yaml \
      containers-prepare-parameter.yaml.orig

   2. Define the following environment variables before you run the script to update the containers-prepare-parameter.yaml file:

      • NAMESPACE: The namespace for the UBI9 images. For example, NAMESPACE='"namespace":"example.redhat.com:5002",'
      • EL8_NAMESPACE: The namespace for the UBI8 images.
      • NEUTRON_DRIVER: The driver to use and determine which OpenStack Networking (neutron) container to use. Set to the type of containers you used to deploy the original stack. For example, set to NEUTRON_DRIVER='"neutron_driver":"ovn",' to use OVN-based containers.

      • EL8_TAGS: The tags of the UBI8 images, for example, EL8_TAGS='"tag":"17.1",'.

        • Replace "17.1", with the tag that you use in your content view.

      • EL9_TAGS: The tags of the UBI9 images, for example, EL9_TAGS='"tag":"17.1",'.

        • Replace "17.1", with the tag that you use in your content view.

          For more information about the tag parameter, see Container image preparation parameters in Customizing your Red Hat OpenStack Platform deployment.

      • CONTROL_PLANE_ROLES: The list of control plane roles using the --role option, for example, --role ControllerOpenstack, --role Database, --role Messaging, --role Networker, --role CephStorage. To view the list of control plane roles in your environment, run the following command:

        $ export STACK=<stack> \
        $ sudo awk '/tripleo_role_name/ {print "--role " $2}' \
        /var/lib/mistral/${STACK}/tripleo-ansible-inventory.yaml \
        | grep -vi compute

        • Replace <stack> with the name of your stack.

      • COMPUTE_ROLES: The list of Compute roles using the --role option, for example, --Compute-1. To view the list of Compute roles in your environment, run the following command:

        $ sudo awk '/tripleo_role_name/ {print "--role " $2}' \
        /var/lib/mistral/${STACK}/tripleo-ansible-inventory.yaml \
        | grep -i compute

      • CEPH_OVERRIDE: If you deployed Red Hat Ceph Storage, specify the Red Hat Ceph Storage 5 container images. For example:

        CEPH_OVERRIDE='"ceph_image":"rhceph-5-rhel8","ceph_tag":"<latest>",'

        • Replace <latest> with the latest ceph_tag version, for example, 5-499.

          The following is an example of the containers-prepare-parameter.yaml file configuration:

          NAMESPACE='"namespace":"registry.redhat.io/rhosp-rhel9",'
          EL8_NAMESPACE='"namespace":"registry.redhat.io/rhosp-rhel8",'
          NEUTRON_DRIVER='"neutron_driver":"ovn",'
          EL8_TAGS='"tag":"17.1",'
          EL9_TAGS='"tag":"17.1",'
          CONTROL_PLANE_ROLES="--role Controller"
          COMPUTE_ROLES="--role Compute1 --role Compute2"
          CEPH_TAGS='"ceph_tag":"5",'

   3. Run the following script to to update the containers-prepare-parameter.yaml file:

      Warning

      If you deployed Red Hat Ceph Storage, ensure that the CEPH_OVERRIDE environment variable is set to the correct values before executing the following command. Failure to do so results in issues when upgrading Red Hat Ceph Storage.

      $ python3 /usr/share/openstack-tripleo-heat-templates/tools/multi-rhel-container-image-prepare.py \
           ${COMPUTE_ROLES} \
           ${CONTROL_PLANE_ROLES} \
           --enable-multi-rhel \
           --excludes collectd \
           --excludes nova-libvirt \
           --minor-override "{${EL8_TAGS}${EL8_NAMESPACE}${CEPH_OVERRIDE}${NEUTRON_DRIVER}\"no_tag\":\"not_used\"}" \
           --major-override "{${EL9_TAGS}${NAMESPACE}${CEPH_OVERRIDE}${NEUTRON_DRIVER}\"no_tag\":\"not_used\"}" \
           --output-env-file \
          /home/stack/containers-prepare-parameter.yaml

      The multi-rhel-container-image-prepare.py script supports the following parameters:

      --output-env-file

      Writes the environment file that contains the default ContainerImagePrepare value.

      --local-push-destination

      Triggers an upload to a local registry.

      --enable-registry-login

      Enables the flag that allows the system to attempt to log in to a remote registry prior to pulling the containers. Use this flag when --local-push-destination is not used and the target systems have network connectivity to remote registries. Do not use this flag for an overcloud that might not have network connectivity to a remote registry.

      --enable-multi-rhel

      Enables multi-rhel.

      --excludes

      Lists the services to exclude.

      --major-override

      Lists the override parameters for a major release.

      --minor-override

      Lists the override parameters for a minor release.

      --role

      The list of roles.

      --role-file

      The role_data.yaml file.

   4. If you deployed Red Hat Ceph Storage, open the containers-prepare-parameter.yaml file to confirm that the Red Hat Ceph Storage 5 container images are specified and that there are no references to Red Hat Ceph Storage 6 container images.

9. If you have a director-deployed Red Hat Ceph Storage deployment, create a file called ceph_params.yaml and include the following content:

   parameter_defaults:
     CephSpecFqdn: true
     CephConfigPath: "/etc/ceph"
     CephAnsibleRepo: "rhceph-5-tools-for-rhel-8-x86_64-rpms"
     DeployedCeph: true
     GrafanaPlugins: []

   Important

   Do not remove the ceph_params.yaml file after the RHOSP upgrade is complete. This file must be present in director-deployed Red Hat Ceph Storage environments. Additionally, any time you run openstack overcloud deploy, you must include the ceph_params.yaml file, for example, -e ceph_params.yaml.

   Note

   If your Red Hat Ceph Storage deployment includes short names, you must set the CephSpecFqdn parameter to false. If set to true, the inventory generates with both the short names and domain names, causing the Red Hat Ceph Storage upgrade to fail.

10. Create an environment file called upgrades-environment.yaml in your templates directory and include the following content:

    parameter_defaults:
      ExtraConfig:
        nova::workarounds::disable_compute_service_check_for_ffu: true
      DnsServers: ["<dns_servers>"]
      DockerInsecureRegistryAddress: [<undercloud_FQDN>]
      UpgradeInitCommand: |
        sudo subscription-manager repos --disable=*
          if $( grep -q  9.2  /etc/os-release )
          then
            sudo subscription-manager repos --enable=rhel-9-for-x86_64-baseos-e4s-rpms --enable=rhel-9-for-x86_64-appstream-e4s-rpms --enable=rhel-9-for-x86_64-highavailability-e4s-rpms --enable=openstack-17.1-for-rhel-9-x86_64-rpms --enable=fast-datapath-for-rhel-9-x86_64-rpms
            sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-9-x86_64-rpms
            sudo subscription-manager release --set=9.2
          else
            sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-aus-rpms --enable=rhel-8-for-x86_64-appstream-aus-rpms --enable=rhel-8-for-x86_64-highavailability-aus-rpms --enable=openstack-17.1-for-rhel-8-x86_64-rpms --enable=fast-datapath-for-rhel-8-x86_64-rpms
            sudo podman ps | grep -q ceph && subscription-manager repos --enable=rhceph-5-tools-for-rhel-8-x86_64-rpms
            sudo subscription-manager release --set=8.4
          fi
    
          if $(sudo podman ps | grep -q ceph )
          then
            sudo dnf -y install cephadm
          fi

    • Replace <dns_servers> with a comma-separated list of your DNS server IP addresses, for example, ["10.0.0.36", "10.0.0.37"].

    • Replace <undercloud_FQDN> with the fully qualified domain name (FQDN) of the undercloud host, for example, ["undercloud-0.ctlplane.redhat.local:8787"].

      For more information about the upgrade parameters that you can configure in the environment file, see Upgrade parameters.

11. On the undercloud, create a file called overcloud_upgrade_prepare.sh in your templates directory. You must create this file for each stack in your environment, starting with the overcloud stack. This file includes the original content of your overcloud deploy file and the environment files that are relevant to each cell stack. For example:

    #!/bin/bash
    openstack overcloud upgrade prepare --yes \
      --timeout 460 \
      --templates /usr/share/openstack-tripleo-heat-templates \
      --ntp-server 192.168.24.1 \
      --stack <overcloud-stack> \
      -r /home/stack/roles_data.yaml \
      -e /home/stack/templates/internal.yaml \
      -e /home/stack/templates/network/network-environment.yaml \
      -e /home/stack/templates/inject-trust-anchor.yaml \
      -e /home/stack/templates/hostnames.yml \
      -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
      -e /home/stack/templates/nodes_data.yaml \
      -e /home/stack/templates/debug.yaml \
      -e /home/stack/templates/firstboot.yaml \
      -e /home/stack/templates/upgrades-environment.yaml \
      -e /home/stack/overcloud-params.yaml \
      -e /usr/share/openstack-tripleo-heat-templates/environments/nova-hw-machine-type-upgrade.yaml \
      -e /home/stack/skip_rhel_release.yaml \
      -e ~/containers-prepare-parameter.yaml \
      -e /home/stack/templates/baremetal-<overcloud-stack>-deployment.yaml \
      -e /home/stack/templates/generated-<overcloud-stack>-networks-deployed.yaml \
      -e /home/stack/templates/generated-<overcloud-stack>-vip-deployed.yaml

    1. In the original network-environment.yaml file (/home/stack/templates/network/network-environment.yaml), remove all the resource_registry resources that point to OS::TripleO::*::Net::SoftwareConfig.

    2. In the overcloud_upgrade_prepare.sh file, include the following options relevant to your environment:

       • The environment file (upgrades-environment.yaml) with the upgrade-specific parameters (-e).
       • The environment file (containers-prepare-parameter.yaml) with your new container image locations (-e). In most cases, this is the same environment file that the undercloud uses.
       • The default_cell_export.yaml file that you exported from the default cell in the overcloud stack. You must add this file to all of the overcloud_upgrade_prepare.sh files for each cell stack, except for the overcloud stack.
       • The environment file (skip_rhel_release.yaml) with the release parameters (-e).
       • Any custom configuration environment files (-e) relevant to your deployment.
       • If applicable, your custom roles (roles_data) file by using --roles-file.
       • For Ceph deployments, the environment file (ceph_params.yaml) with the Ceph parameters (-e).
       • If applicable, the environment file (ipa-environment.yaml) with your IPA service (-e).
       • If you are using composable networks, the (network_data) file by using --network-file.
       • The files that were generated during overcloud adoption (networks-deployed.yaml, vip-deployed.yaml,baremetal-deployment.yaml) (-e). These files must be placed last in the overcloud upgrade prepare script.

       • If you use OVS, add the OVS environment files in the following order:

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-sriov.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs-dpdk.yaml

       • If you use OVN, add the OVN environment files in the following order:

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-dpdk.yaml
         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-sriov.yaml

         • /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovn-ha.yaml

           Note

           Do not include the network-isolation.yaml file in your overcloud deploy file or the overcloud_upgrade_prepare.sh file. Network isolation is defined in the network_data.yaml file.

       • If you use a custom stack name, pass the name with the --stack option.

         Note

         You must include the nova-hw-machine-type-upgrade.yaml file in your templates until all of your RHEL 8 Compute nodes are upgraded to RHEL 9 in the environment. If this file is excluded, an error appears in the nova_compute.log in the /var/log/containers/nova directory. After you upgrade all of your RHEL 8 Compute nodes to RHEL 9, you can remove this file from your configuration and update the stack.

    3. In the director-deployed Red Hat Ceph Storage use case, if you enabled the Shared File Systems service (manila) with CephFS through NFS on the deployment that you are upgrading, you must specify an additional environment file at the end of the overcloud_upgrade_prepare.sh script file. You must add the environment file at the end of the script because it overrides another environment file that is specified earlier in the script:

       -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/manila-cephfsganesha-config.yaml

    4. In the external Red Hat Ceph Storage use case, if you enabled the Shared File Systems service (manila) with CephFS through NFS on the deployment that you are upgrading, you must check that the associated environment file in the overcloud_upgrade_prepare.sh script points to the tripleo-based ceph-nfs role. If present, remove the following environment file:

       -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/manila-cephfsganesha-config.yaml

       And add the following environment file:

       -e /usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml

12. Run the upgrade preparation script on the overcloud stack:

    $ source stackrc
    $ chmod 755 /home/stack/overcloud_upgrade_prepare.sh
    $ sh /home/stack/overcloud_upgrade_prepare.sh

    Wait until the upgrade preparation completes.

13. Export the cell configuration and password information from the default cell in the overcloud stack to a new common environment file for the multi-cell deployment. You must include the /home/stack/common/default_cell_export.yaml file and all of the environment files that are relevant to your environment in the script for each cell:

    (undercloud) $ openstack overcloud cell export --control-plane-stack overcloud \
     -f --output-file common/default_cell_export.yaml \
     --working-dir /home/stack/overcloud-deploy/overcloud/

14. If you have a DCN environment, the generated cell_export.yaml file is incomplete. To get the correct file, complete the following steps:

    1. Source the stackrc undercloud credentials file:

       $ source ~/stackrc

    2. Force the necessary data to be generated into the stack’s overcloud-deploy directory:

       $ openstack overcloud upgrade run --yes --limit allovercloud,undercloud --stack overcloud --playbook overcloud/upgrade_steps_playbook.yaml --skip-tags upgrade_steps

    3. Re-export the overcloud stack to to get the updated cell_export.yaml file to use in the overcloud upgrade prepare script:

       $ openstack overcloud cell export --control-plane-stack overcloud --force-overwrite --working-dir overcloud-deploy/overcloud --output-file overcloud-deploy/overcloud/overcloud-cell-export.yaml

15. Run the upgrade preparation script on each cell stack. For example:

    $ source stackrc
    $ chmod 755 /home/stack/<stack1>-overcloud_upgrade_prepare.sh
    $ sh /home/stack/<stack1>-overcloud_upgrade_prepare.sh

    Wait until the upgrade preparation completes.

16. Download the container images for each cell stack, starting with the overcloud stack:

    $ openstack overcloud external-upgrade run --stack <stack> --tags container_image_prepare