Chapter 4. Preparing for the Overcloud Upgrade

Procedure

1. Edit the environment file containing your registration details. For example:

   $ vi ~/templates/rhel-registration/environment-rhel-registration.yaml

   Copy to Clipboard Toggle word wrap

2. Edit the following parameter values:

   rhel_reg_repos

   Update to include the new repositories for Red Hat OpenStack Platform 12.

   rhel_reg_activation_key

   Update the activation key to access the Red Hat OpenStack Platform 12 repositories.

   rhel_reg_sat_repo

   If using a newer version of Red Hat Satellite 6, update the repository containing Satellite 6’s management tools.

3. Save the environment file.

Procedure

1. Install the ceph-ansible package to the undercloud:

   [stack@director ~]$ sudo yum install -y ceph-ansible

   Copy to Clipboard Toggle word wrap

2. Check that you are using the latest resources and configuration in your storage environment file. This requires the following changes:

   1. The resource_registry uses containerized services from the docker/services subdirectory of your core Heat template collection. For example:

1. Use the new CephAnsibleDisksConfig parameter to define how your disks are mapped. Previous versions of Red Hat OpenStack Platform used the ceph::profile::params::osds hieradata to define the OSD layout. Convert this hieradata to the structure of the CephAnsibleDisksConfig parameter. For example, if your hieradata contained the following:

   parameter_defaults:
     ExtraConfig:
       ceph::profile::params::osd_journal_size: 512
       ceph::profile::params::osds:
         '/dev/sdb': {}
         '/dev/sdc': {}
         '/dev/sdd': {}

   Copy to Clipboard Toggle word wrap

   Then the CephAnsibleDisksConfig would look like this:

   parameters_default:
     CephAnsibleDisksConfig:
       devices:
       - /dev/sdb
       - /dev/sdc
       - /dev/sdd
       journal_size: 512
       osd_scenario: collocated

   Copy to Clipboard Toggle word wrap

   For a full list of OSD disk layout options used in ceph-ansible, view the sample file in /usr/share/ceph-ansible/group_vars/osds.yml.sample.

Procedure

1. Edit the environment file containing your Ceph Storage configuration.

2. Ensure that the resource_registry uses the Puppet resources. For example:

   resource_registry:
     OS::TripleO::Services::CephMon: ../puppet/services/ceph-mon.yaml
     OS::TripleO::Services::CephOSD: ../puppet/services/ceph-osd.yaml
     OS::TripleO::Services::CephClient: ../puppet/services/ceph-client.yaml

   Copy to Clipboard Toggle word wrap
   Note

   Use the contents of the /usr/share/openstack-tripleo-heat-templates/environments/puppet-ceph.yaml as an example.

3. Upgrade your Controller-based nodes to containerized services using the instructions in Section 5.1, “Upgrading the Overcloud Nodes”.
4. Upgrade your HCI nodes using the instructions in Section 5.3, “Upgrading the Compute Nodes”

5. Edit the resource_registry in your Ceph Storage configuration to use the containerized services:

   resource_registry:
     OS::TripleO::Services::CephMon: ../docker/services/ceph-ansible/ceph-mon.yaml
     OS::TripleO::Services::CephOSD: ../docker/services/ceph-ansible/ceph-osd.yaml
     OS::TripleO::Services::CephClient: ../docker/services/ceph-ansible/ceph-client.yaml

   Copy to Clipboard Toggle word wrap
   Note

   Use the contents of the /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml as an example.

6. Add the CephAnsiblePlaybook parameter to the parameter_defaults section of your storage environment file:

     CephAnsiblePlaybook: /usr/share/ceph-ansible/infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml

   Copy to Clipboard Toggle word wrap

7. Add the CephAnsibleDisksConfig parameter to the parameter_defaults section of your storage environment file and define the disk layout. For example:

     CephAnsibleDisksConfig:
       devices:
       - /dev/vdb
       - /dev/vdc
       - /dev/vdd
       journal_size: 512
       osd_scenario: collocated

   Copy to Clipboard Toggle word wrap
8. Finalize the upgrade of your overcloud using the instructions in Section 5.4, “Finalizing the Upgrade”.

Procedure

1. The director’s dynamic Ansible script has updated to the OpenStack Platform 12 version, which uses the RoleNetHostnameMap Heat parameter in the overcloud plan to define the inventory. However, the overcloud currently uses the OpenStack Platform 11 template versions, which do not have the RoleNetHostnameMap parameter. This means you need to create a temporary static inventory file, which you can generate with the following command:

   $ openstack server list -c Networks -f value | cut -d"=" -f2 > overcloud_hosts

   Copy to Clipboard Toggle word wrap

2. Create an Ansible playbook (undercloud-ca.yml) that contains the following:

   ---
   - name: Add undercloud CA to overcloud nodes
     hosts: all
     user: heat-admin
     become: true
     tasks:
       - name: Copy undercloud CA
         copy:
           src: ca.crt.pem
           dest: /etc/pki/ca-trust/source/anchors/
       - name: Update trust
         command: "update-ca-trust extract"
       - name: Get the hostname of the undercloud
         delegate_to: 127.0.0.1
         command: hostname
         register: undercloud_hostname
       - name: Verify URL
         uri:
           url: https://{{ undercloud_hostname.stdout }}:13808/healthcheck
           return_content: yes
         register: verify
       - name: Report output
         debug:
           msg: "{{ ansible_hostname }} can access the undercloud's Public API"
         when: verify.content == "OK"

   Copy to Clipboard Toggle word wrap

   This playbook contains multiple tasks that perform the following on each node:

   • Copy the undercloud’s certificate authority file (ca.crt.pem) to the overcloud node. The name of this file and its location might vary depending on your configuration. This example uses the name and location defined during the self-signed certificate procedure (see "SSL/TLS Certificate Configuration" in the Director Installation and Usage guide).
   • Execute the command to update the certificate authority trust database on the overcloud node.
   • Checks the undercloud’s Object Storage Public API from the overcloud node and reports if successful.

3. Run the playbook with the following command:

   $ ansible-playbook -i overcloud_hosts undercloud-ca.yml

   Copy to Clipboard Toggle word wrap

   This uses the temporary inventory to provide Ansible with your overcloud nodes.

4. The resulting Ansible output should show a debug message for node. For example:

   ok: [192.168.24.100] => {
       "msg": "overcloud-controller-0 can access the undercloud's Public API"
   }

   Copy to Clipboard Toggle word wrap

Procedure

1. Run the following commands to save a list of node IP addresses in the OVERCLOUD_HOSTS environment variable:

   $ source ~/stackrc
   $ export OVERCLOUD_HOSTS=$(openstack server list -f value -c Networks | cut -d "=" -f 2 | tr '\n' ' ')

   Copy to Clipboard Toggle word wrap

2. Run the following script:

   $ /usr/share/openstack-tripleo-heat-templates/deployed-server/scripts/enable-ssh-admin.sh

   Copy to Clipboard Toggle word wrap
3. Proceed with the upgrade.

4. When upgrading Compute or Object Storage nodes, use the following:

   1. Use the -U option with the upgrade-non-controller.sh script and specify the stack user. This is because the default user for pre-provisioned nodes is stack and not heat-admin.

   2. Use the node’s IP address with the --upgrade option. This is because the node are not managed with the director’s Compute (nova) and Bare Metal (ironic) services and do not have a node name.

      For example:

      $ upgrade-non-controller.sh -U stack --upgrade 192.168.24.100

      Copy to Clipboard Toggle word wrap

1. In the parameter_defaults section, add a network deployment parameter to run os-net-config during the upgrade process to associate the OVS 2.7 PCI address with DPDK ports:

   parameter_defaults:
     ComputeNetworkDeploymentActions: ['CREATE', 'UPDATE']

   Copy to Clipboard Toggle word wrap

   The parameter name must match the name of the role you use to deploy DPDK. In this example, the role name is Compute so the parameter name is ComputeNetworkDeploymentActions.

   Note

   This parameter is not needed after the initial upgrade and should be removed from the environment file.

2. In the resource_registry section, override the ComputeNeutronOvsAgent service to the neutron-ovs-dpdk-agent puppet service:

   resource_registry:
     OS::TripleO::Services::ComputeNeutronOvsAgent: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-ovs-dpdk-agent.yaml

   Copy to Clipboard Toggle word wrap

   Red Hat OpenStack Platform 12 added a new service (OS::TripleO::Services::ComputeNeutronOvsDpdk) to support the addition of the new ComputeOvsDpdk role. The example above maps this externally for upgrades.