Chapter 1. Upgrade a Red Hat Ceph Storage cluster using cephadm

Procedure

1. Enable the Ceph Ansible repositories on the Ansible administration node:

   Red Hat Enterprise Linux 9

   subscription-manager repos --enable=rhceph-7-tools-for-rhel-9-x86_64-rpms

2. Update the cephadm and cephadm-ansible package:

   Example

   [root@admin ~]# dnf update cephadm
   [root@admin ~]# dnf update cephadm-ansible

3. Navigate to the /usr/share/cephadm-ansible/ directory:

   Example

   [root@admin ~]# cd /usr/share/cephadm-ansible

4. Run the preflight playbook with the upgrade_ceph_packages parameter set to true on the bootstrapped host in the storage cluster:

   Syntax

   ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=rhcs upgrade_ceph_packages=true"

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i /etc/ansible/hosts cephadm-preflight.yml --extra-vars "ceph_origin=rhcs upgrade_ceph_packages=true"

   This package upgrades cephadm on all the nodes.

5. Log into the cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

6. Ensure all the hosts are online and that the storage cluster is healthy:

   Example

   [ceph: root@host01 /]# ceph -s

7. Set the OSD noout, noscrub, and nodeep-scrub flags to prevent OSDs from getting marked out during upgrade and to avoid unnecessary load on the cluster:

   Example

   [ceph: root@host01 /]# ceph osd set noout
   [ceph: root@host01 /]# ceph osd set noscrub
   [ceph: root@host01 /]# ceph osd set nodeep-scrub

8. Check service versions and the available target containers:

   Syntax

   ceph orch upgrade check IMAGE_NAME

   Example

   [ceph: root@host01 /]# ceph orch upgrade check registry.redhat.io/rhceph/rhceph-7-rhel9:latest

   Note

   The image name is applicable for both Red Hat Enterprise Linux 8 and Red Hat Enterprise Linux 9.

9. Upgrade the storage cluster:

   Syntax

   ceph orch upgrade start IMAGE_NAME

   Example

   [ceph: root@host01 /]# ceph orch upgrade start registry.redhat.io/rhceph/rhceph-7-rhel9:latest

   Note

   To perform a staggered upgrade, see Performing a staggered upgrade.

   While the upgrade is underway, a progress bar appears in the ceph status output.

   Example

   [ceph: root@host01 /]# ceph status
   [...]
   progress:
       Upgrade to 18.2.0-128.el9cp (1s)
         [............................]

10. Verify the new IMAGE_ID and VERSION of the Ceph cluster:

    Example

    [ceph: root@host01 /]# ceph versions
    [ceph: root@host01 /]# ceph orch ps

    Note

    If you are not using the cephadm-ansible playbooks, after upgrading your Ceph cluster, you must upgrade the ceph-common package and client libraries on your client nodes.

    Example

    [root@client01 ~] dnf update ceph-common

    Verify you have the latest version:

    Example

    [root@client01 ~] ceph --version

11. When the upgrade is complete, unset the noout, noscrub, and nodeep-scrub flags:

    Example

    [ceph: root@host01 /]# ceph osd unset noout
    [ceph: root@host01 /]# ceph osd unset noscrub
    [ceph: root@host01 /]# ceph osd unset nodeep-scrub

Procedure

1. Enable the Ceph Ansible repositories on the Ansible administration node:

   Red Hat Enterprise Linux 9

   subscription-manager repos --enable=rhceph-7-tools-for-rhel-9-x86_64-rpms

2. Update the cephadm and cephadm-ansible package.

   Example

   [root@admin ~]# dnf update cephadm
   [root@admin ~]# dnf update cephadm-ansible

3. Navigate to the /usr/share/cephadm-ansible/ directory:

   Example

   [root@admin ~]# cd /usr/share/cephadm-ansible

4. Run the preflight playbook with the upgrade_ceph_packages parameter set to true and the ceph_origin parameter set to custom on the bootstrapped host in the storage cluster:

   Syntax

   ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true"

   Example

   [ceph-admin@admin ~]$ ansible-playbook -i /etc/ansible/hosts cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true"

   This package upgrades cephadm on all the nodes.

5. Log into the cephadm shell:

   Example

   [root@node0 ~]# cephadm shell

6. Ensure all the hosts are online and that the storage cluster is healthy:

   Example

   [ceph: root@node0 /]# ceph -s

7. Set the OSD noout, noscrub, and nodeep-scrub flags to prevent OSDs from getting marked out during upgrade and to avoid unnecessary load on the cluster:

   Example

   [ceph: root@host01 /]# ceph osd set noout
   [ceph: root@host01 /]# ceph osd set noscrub
   [ceph: root@host01 /]# ceph osd set nodeep-scrub

8. Check service versions and the available target containers:

   Syntax

   ceph orch upgrade check IMAGE_NAME

   Example

   [ceph: root@node0 /]# ceph orch upgrade check LOCAL_NODE_FQDN:5000/rhceph/rhceph-7-rhel9

9. Upgrade the storage cluster:

   Syntax

   ceph orch upgrade start IMAGE_NAME

   Example

   [ceph: root@node0 /]# ceph orch upgrade start LOCAL_NODE_FQDN:5000/rhceph/rhceph-7-rhel9

   While the upgrade is underway, a progress bar appears in the ceph status output.

   Example

   [ceph: root@node0 /]# ceph status
   [...]
   progress:
       Upgrade to 18.2.0-128.el9cp (1s)
         [............................]

10. Verify the new IMAGE_ID and VERSION of the Ceph cluster:

    Example

    [ceph: root@node0 /]# ceph version
    [ceph: root@node0 /]# ceph versions
    [ceph: root@node0 /]# ceph orch ps

11. When the upgrade is complete, unset the noout, noscrub, and nodeep-scrub flags:

    Example

    [ceph: root@host01 /]# ceph osd unset noout
    [ceph: root@host01 /]# ceph osd unset noscrub
    [ceph: root@host01 /]# ceph osd unset nodeep-scrub