Chapter 16. Upgrading Red Hat Ceph Storage 5 to 6

Procedure

1. Create a playbook to enable the Red Hat Ceph Storage (tool only) repositories in the Controller nodes. It should contain the following information:

   - hosts: all
     gather_facts: false
     tasks:
       - name: Enable RHCS 6 tools repo
         ansible.builtin.command: |
             subscription-manager repos --disable=rhceph-5-tools-for-rhel-9-x86_64-rpms --enable=rhceph-6-tools-for-rhel-9-x86_64-rpms
         become: true
       - name: Update cephadm
         ansible.builtin.package:
           name: cephadm
           state: latest
         become: true

   Copy to Clipboard Toggle word wrap

2. Run the playbook:

   ansible-playbook -i ~/overcloud-deploy/<stack>/tripleo-ansible-inventory.yaml <playbook_file_name> --limit <controller_role>

   • Replace <stack> with the name of your stack.
   • Replace <playbook_file_name> with the name of the playbook created in the previous step.
   • Replace <controller_role> with the role applied to Controller nodes.
   • Use the --limit option to apply the content to Controller nodes only.
3. Log in to a Controller node.

4. Verify that the cephadm package is updated to Release 6:

   $ sudo dnf info cephadm | grep -i version

Procedure

1. Locate your container preparation file. The default name of this file is containers-prepare-parameter.yaml.
2. Edit the container preparation file.

3. Locate the ceph_tag parameter. The current entry should be similar to the following example:

   ceph_namespace: registry.redhat.io
   ceph_image: rhceph-6-rhel9
   ceph_tag: '5'

   Copy to Clipboard Toggle word wrap

4. Update the ceph_tag parameter for Red Hat Ceph Storage 6:

   ceph_namespace: registry.redhat.io
   ceph_image: rhceph-6-rhel9
   ceph_tag: '6'

   Copy to Clipboard Toggle word wrap

5. Edit the containers-image-prepare.yaml file and replace the Red Hat Ceph monitoring stack container related parameters with the following content:

   ceph_alertmanager_image: ose-prometheus-alertmanager
   ceph_alertmanager_namespace: registry.redhat.io/openshift4
   ceph_alertmanager_tag: v4.12
   ceph_grafana_image: rhceph-6-dashboard-rhel9
   ceph_grafana_namespace: registry.redhat.io/rhceph
   ceph_grafana_tag: latest
   ceph_node_exporter_image: ose-prometheus-node-exporter
   ceph_node_exporter_namespace: registry.redhat.io/openshift4
   ceph_node_exporter_tag: v4.12
   ceph_prometheus_image: ose-prometheus
   ceph_prometheus_namespace: registry.redhat.io/openshift4
   ceph_prometheus_tag: v4.12

   Copy to Clipboard Toggle word wrap
6. Save the file.

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap

3. Run the container preparation command:

   $ openstack tripleo container image prepare -e <container_preparation_file>

   • Replace <container_preparation_file> with the name of your file. The default file is containers-prepare-parameter.yaml.

4. Verify that the new Red Hat Ceph Storage image is present in the undercloud registry:

   $ openstack tripleo container image list -f value | awk -F '//' '/ceph/ {print $2}'

5. Verify that the new Red Hat Ceph Storage monitoring stack images are present in the undercloud registry:

   $ openstack tripleo container image list -f value | awk -F '//' '/dashboard|grafana|prometheus|alertmanager|node-exporter/ {print $2}'

Procedure

1. Log in to a Controller node.

2. Upgrade the cluster to the latest Red Hat Ceph Storage version by using Upgrade a Red Hat Ceph Storage cluster using cephadm in the Red Hat Ceph Storage 6 Upgrade Guide.

   Important

   Steps 1 to 4 of the upgrade procedure presented in Upgrading the Red Hat Ceph Storage cluster are not necessary when upgrading a Red Hat Ceph Storage cluster deployed with Red Hat OpenStack by director. Start at step 5 of the procedure.

3. Wait until the Red Hat Ceph Storage container upgrade completes. Monitor the status of the upgrade by using the following command:

   $ sudo cephadm shell -- ceph orch upgrade status

   Copy to Clipboard Toggle word wrap

Procedure

1. Log in to a Controller node.

2. Inspect the ceph-nfs service:

   $ sudo pcs status | grep ceph-nfs

3. Inspect the ceph-nfs systemd unit to confirm that it contains the Red Hat Ceph Storage 5 container image and tag:

   $ cat /etc/systemd/system/ceph-nfs@.service | grep -i container_image

   Copy to Clipboard Toggle word wrap

4. Create a file called /home/stack/ganesha_update_extravars.yaml with the following content:

   tripleo_cephadm_container_image: <ceph_image_name>
   tripleo_cephadm_container_ns: <ceph_image_namespace>
   tripleo_cephadm_container_tag: <ceph_image_tag>

   Copy to Clipboard Toggle word wrap
   • Replace <ceph_image_name> with the name of the Red Hat Ceph Storage container image.
   • Replace <ceph_image_namespace> with the name of the Red Hat Ceph Storage container namespace.

   • Replace <ceph_image_tag> with the name of the Red Hat Ceph Storage container tag.

     For example, in a typical environment, this content would have the following values:

     tripleo_cephadm_container_image: rhceph-6-rhel9
     tripleo_cephadm_container_ns: undercloud-0.ctlplane.redhat.local:8787
     tripleo_cephadm_container_tag: '6'

     Copy to Clipboard Toggle word wrap
5. Save the file.

6. Run the ceph-update-ganesha.yml playbook and provide the ganesha_update_extravars.yaml playbook for additional command parameters:

   ansible-playbook -i $HOME/overcloud-deploy/<stack>/config-download/<stack>/tripleo-ansible-inventory.yaml \
       /usr/share/ansible/tripleo-playbooks/ceph-update-ganesha.yml \
        -e @$HOME/overcloud-deploy/<stack>/config-download/<stack>/global_vars.yaml \
        -e @$HOME/overcloud-deploy/<stack>/config-download/<stack>/cephadm/cephadm-extra-vars-heat.yml \
        -e @$HOME/ganesha_update_extravars.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the overcloud stack.

7. Verify that the ceph-nfs service is running:

   $ sudo pcs status | grep ceph-nfs

8. Verify that the ceph-nfs systemd unit contains the Red Hat Ceph Storage 6 container image and tag:

   $ cat /etc/systemd/system/ceph-nfs@.service | grep rhceph

Procedure

1. Locate your container preparation file. The default name of this file is containers-prepare-parameter.yaml.
2. Edit the container preparation file.

3. Locate the ceph_tag parameter. The current entry should be similar to the following example:

   ceph_namespace: registry.redhat.io
   ceph_image: rhceph-6-rhel9
   ceph_tag: '5'

   Copy to Clipboard Toggle word wrap

4. Update the ceph_tag parameter for Red Hat Ceph Storage 6:

   ceph_namespace: registry.redhat.io
   ceph_image: rhceph-6-rhel9
   ceph_tag: '6'

   Copy to Clipboard Toggle word wrap

5. Edit the containers-image-prepare.yaml file and replace the Red Hat Ceph monitoring stack container related parameters with the following content:

   ceph_alertmanager_image: ose-prometheus-alertmanager
   ceph_alertmanager_namespace: registry.redhat.io/openshift4
   ceph_alertmanager_tag: v4.12
   ceph_grafana_image: rhceph-6-dashboard-rhel9
   ceph_grafana_namespace: registry.redhat.io/rhceph
   ceph_grafana_tag: latest
   ceph_node_exporter_image: ose-prometheus-node-exporter
   ceph_node_exporter_namespace: registry.redhat.io/openshift4
   ceph_node_exporter_tag: v4.12
   ceph_prometheus_image: ose-prometheus
   ceph_prometheus_namespace: registry.redhat.io/openshift4
   ceph_prometheus_tag: v4.12

   Copy to Clipboard Toggle word wrap
6. Save the file.

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the stackrc undercloud credentials file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap

3. Run the container preparation command:

   $ openstack tripleo container image prepare -e <container_preparation_file>

   • Replace <container_preparation_file> with the name of your file. The default file is containers-prepare-parameter.yaml.

4. Verify that the new Red Hat Ceph Storage image is present in the undercloud registry:

   $ openstack tripleo container image list -f value | awk -F '//' '/ceph/ {print $2}'

5. Verify that the new Red Hat Ceph Storage monitoring stack images are present in the undercloud registry:

   $ openstack tripleo container image list -f value | awk -F '//' '/dashboard|grafana|prometheus|alertmanager|node-exporter/ {print $2}'

Procedure

1. Log in to a Controller node.

2. Inspect the ceph-nfs service:

   $ sudo pcs status | grep ceph-nfs

3. Inspect the ceph-nfs systemd unit to confirm that it contains the Red Hat Ceph Storage 5 container image and tag:

   $ cat /etc/systemd/system/ceph-nfs@.service | grep -i container_image

   Copy to Clipboard Toggle word wrap

4. Create a file called /home/stack/ganesha_update_extravars.yaml with the following content:

   tripleo_cephadm_container_image: <ceph_image_name>
   tripleo_cephadm_container_ns: <ceph_image_namespace>
   tripleo_cephadm_container_tag: <ceph_image_tag>

   Copy to Clipboard Toggle word wrap
   • Replace <ceph_image_name> with the name of the Red Hat Ceph Storage container image.
   • Replace <ceph_image_namespace> with the name of the Red Hat Ceph Storage container namespace.

   • Replace <ceph_image_tag> with the name of the Red Hat Ceph Storage container tag.

     For example, in a typical environment, this content would have the following values:

     tripleo_cephadm_container_image: rhceph-6-rhel9
     tripleo_cephadm_container_ns: undercloud-0.ctlplane.redhat.local:8787
     tripleo_cephadm_container_tag: '6'

     Copy to Clipboard Toggle word wrap
5. Save the file.

6. Run the ceph-update-ganesha.yml playbook and provide the ganesha_update_extravars.yaml playbook for additional command parameters:

   ansible-playbook -i $HOME/overcloud-deploy/<stack>/config-download/<stack>/tripleo-ansible-inventory.yaml \
       /usr/share/ansible/tripleo-playbooks/ceph-update-ganesha.yml \
        -e @$HOME/overcloud-deploy/<stack>/config-download/<stack>/global_vars.yaml \
        -e @$HOME/overcloud-deploy/<stack>/config-download/<stack>/cephadm/cephadm-extra-vars-heat.yml \
        -e @$HOME/ganesha_update_extravars.yaml

   Copy to Clipboard Toggle word wrap
   • Replace <stack> with the name of the overcloud stack.

7. Verify that the ceph-nfs service is running:

   $ sudo pcs status | grep ceph-nfs

8. Verify that the ceph-nfs systemd unit contains the Red Hat Ceph Storage 6 container image and tag:

   $ cat /etc/systemd/system/ceph-nfs@.service | grep rhceph