Red Hat Summit Red Hat SummitSupportConsoleDéveloppeursCommencer un essaiContact

Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 1. Upgrade a Red Hat Ceph Storage cluster using cephadm

As a storage administrator, you can use the cephadm Orchestrator to upgrade Red Hat Ceph Storage 5 and later.

Note

Upgrading directly from Red Hat Ceph Storage 4 to Red Hat Ceph Storage 6 is not supported.

The automated upgrade process follows Ceph best practices. For example:

  • The upgrade order starts with Ceph Managers, Ceph Monitors, then other daemons.
  • Each daemon is restarted only after Ceph indicates that the cluster will remain available.

The storage cluster health status is likely to switch to HEALTH_WARNING during the upgrade. When the upgrade is complete, the health status should switch back to HEALTH_OK.

Warning

If you have a Red Hat Ceph Storage 6 cluster with multi-site configured, do not upgrade to the latest version of 6.1.z1 as there are issues with data corruption on encrypted objects when objects replicate to the disaster recovery (DR) site.

Note

You do not get a message once the upgrade is successful. Run ceph versions and ceph orch ps commands to verify the new image ID and the version of the storage cluster.

Warning

The Ceph iSCSI gateway is removed from Red Hat Ceph Storage 6. Therefore, you need to manage the iSCSI LUNs before upgrading from Red Hat Ceph Storage 5 to Red Hat Ceph Storage 6.

When you upgrade a Red Hat Ceph Storage cluster from RHCS 5 to RHCS 6, RBD images that were exported through iSCSI are preserved, therefore data is not lost. However, because all iSCSI targets disappear with the upgrade, data is temporarily inaccessible. To recover data, you can map such RBD images with rbd device map command or export them to a file with rbd export command.

podman and Red Hat Ceph Storage have different end-of-life strategies that might make it challenging to find compatible versions.

Red Hat recommends to use the podman version shipped with the corresponding Red Hat Enterprise Linux version for Red Hat Ceph Storage. See the Red Hat Ceph Storage: Supported configurations knowledge base article for more details. See the Contacting Red Hat support for service section in the Red Hat Ceph Storage Troubleshooting Guide for additional assistance.

The following table shows version compatibility between Red Hat Ceph Storage 6 and versions of podman.

Expand
CephPodman     
 

1.9

2.0

2.1

2.2

3.0

>3.0

Red Hat Ceph Storage 6

false

true

true

false

true

true

Warning

To use Podman with Red Hat Ceph Storage 5 and later, you must use a version of Podman that is 2.0.0 or higher.

1.2. Upgrading the Red Hat Ceph Storage cluster
Copier lien

You can use ceph orch upgrade command for upgrading a Red Hat Ceph Storage 5.0 cluster.

Prerequisites

  • A running Red Hat Ceph Storage cluster 5.
  • Red Hat Enterprise Linux 9.0 or later with ansible-core bundled into AppStream.
  • Root-level access to all the nodes.
  • Ansible user with sudo and passwordless ssh access to all nodes in the storage cluster.
  • At least two Ceph Manager nodes in the storage cluster: one active and one standby.
Important

Upgrade to the latest version of Red Hat Ceph Storage 5.3.z5 prior to upgrading to the latest version Red Hat Ceph Storage 6.1.

Note

Red Hat Ceph Storage 5 also includes a health check function that returns a DAEMON_OLD_VERSION warning if it detects that any of the daemons in the storage cluster are running multiple versions of RHCS. The warning is triggered when the daemons continue to run multiple versions of Red Hat Ceph Storage beyond the time value set in the mon_warn_older_version_delay option. By default, the mon_warn_older_version_delay option is set to 1 week. This setting allows most upgrades to proceed without falsely seeing the warning. If the upgrade process is paused for an extended time period, you can mute the health warning: 

ceph health mute DAEMON_OLD_VERSION --sticky
Copy to Clipboard Toggle word wrap

After the upgrade has finished, unmute the health warning: 

ceph health unmute DAEMON_OLD_VERSION
Copy to Clipboard Toggle word wrap

Procedure

  1. Enable the Ceph Ansible repositories on the Ansible administration node:

    Red Hat Enterprise Linux 9

    subscription-manager repos --enable=rhceph-6-tools-for-rhel-9-x86_64-rpms
    Copy to Clipboard Toggle word wrap

  2. Update the cephadm and cephadm-ansible package:

    Example

    [root@admin ~]# dnf update cephadm
[root@admin ~]# dnf update cephadm-ansible
    Copy to Clipboard Toggle word wrap

  3. Navigate to the /usr/share/cephadm-ansible/ directory:

    Example

    [root@admin ~]# cd /usr/share/cephadm-ansible
    Copy to Clipboard Toggle word wrap

  4. Run the preflight playbook with the upgrade_ceph_packages parameter set to true on the bootstrapped host in the storage cluster:

    Syntax

    ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=rhcs upgrade_ceph_packages=true"
    Copy to Clipboard Toggle word wrap

    Example

    [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i /etc/ansible/hosts cephadm-preflight.yml --extra-vars "ceph_origin=rhcs upgrade_ceph_packages=true"
    Copy to Clipboard Toggle word wrap

    This package upgrades cephadm on all the nodes.

  5. Log into the cephadm shell:

    Example

    [root@host01 ~]# cephadm shell
    Copy to Clipboard Toggle word wrap

  6. Ensure all the hosts are online and that the storage cluster is healthy:

    Example

    [ceph: root@host01 /]# ceph -s
    Copy to Clipboard Toggle word wrap

  7. Set the OSD noout, noscrub, and nodeep-scrub flags to prevent OSDs from getting marked out during upgrade and to avoid unnecessary load on the cluster:

    Example

    [ceph: root@host01 /]# ceph osd set noout
[ceph: root@host01 /]# ceph osd set noscrub
[ceph: root@host01 /]# ceph osd set nodeep-scrub
    Copy to Clipboard Toggle word wrap

  8. Check service versions and the available target containers:

    Syntax

    ceph orch upgrade check IMAGE_NAME
    Copy to Clipboard Toggle word wrap

    Example

    [ceph: root@host01 /]# ceph orch upgrade check registry.redhat.io/rhceph/rhceph-6-rhel9:latest
    Copy to Clipboard Toggle word wrap

    Note

    The image name is applicable for both Red Hat Enterprise Linux 8 and Red Hat Enterprise Linux 9.

  9. Upgrade the storage cluster:

    Syntax

    ceph orch upgrade start IMAGE_NAME
    Copy to Clipboard Toggle word wrap

    Example

    [ceph: root@host01 /]# ceph orch upgrade start registry.redhat.io/rhceph/rhceph-6-rhel9:latest
    Copy to Clipboard Toggle word wrap

    Note

    To perform a staggered upgrade, see Performing a staggered upgrade.

    While the upgrade is underway, a progress bar appears in the ceph status output.

    Example

    [ceph: root@host01 /]# ceph status
[...]
progress:
    Upgrade to 17.2.6-70.el9cp (1s)
      [............................]
    Copy to Clipboard Toggle word wrap

  10. Verify the new IMAGE_ID and VERSION of the Ceph cluster:

    Example

    [ceph: root@host01 /]# ceph versions
[ceph: root@host01 /]# ceph orch ps
    Copy to Clipboard Toggle word wrap

    Note

    If you are not using the cephadm-ansible playbooks, after upgrading your Ceph cluster, you must upgrade the ceph-common package and client libraries on your client nodes.

    Example

    [root@client01 ~] dnf update ceph-common
    Copy to Clipboard Toggle word wrap

    Verify you have the latest version:

    Example

    [root@client01 ~] ceph --version
    Copy to Clipboard Toggle word wrap

  11. When the upgrade is complete, unset the noout, noscrub, and nodeep-scrub flags:

    Example

    [ceph: root@host01 /]# ceph osd unset noout
[ceph: root@host01 /]# ceph osd unset noscrub
[ceph: root@host01 /]# ceph osd unset nodeep-scrub
    Copy to Clipboard Toggle word wrap

You can upgrade the storage cluster in a disconnected environment by using the --image tag.

You can use ceph orch upgrade command for upgrading a Red Hat Ceph Storage 5 cluster.

Important

Red Hat Enterprise Linux 9 and later does not support the cephadm-ansible playbook.

Prerequisites

  • A running Red Hat Ceph Storage cluster 5.
  • Red Hat Enterprise Linux 9.0 or later with ansible-core bundled into AppStream.
  • Root-level access to all the nodes.
  • Ansible user with sudo and passwordless ssh access to all nodes in the storage cluster.
  • At least two Ceph Manager nodes in the storage cluster: one active and one standby.
  • Register the nodes to CDN and attach subscriptions.
  • Check for the customer container images in a disconnected environment and change the configuration, if required. See the Changing configurations of custom container images for disconnected installations section in the Red Hat Ceph Storage Installation Guide for more details.

By default, the monitoring stack components are deployed based on the primary Ceph image. For disconnected environment of the storage cluster, you have to use the latest available monitoring stack component images.

Expand
Table 1.1. Custom image details for monitoring stack
Monitoring stack componentImage details

Prometheus

registry.redhat.io/openshift4/ose-prometheus:v4.12

Grafana

registry.redhat.io/rhceph-6-dashboard-rhel9:latest

Node-exporter

registry.redhat.io/openshift4/ose-prometheus-node-exporter:v4.12

AlertManager

registry.redhat.io/openshift4/ose-prometheus-alertmanager:v4.12

HAProxy

registry.redhat.io/rhceph/rhceph-haproxy-rhel9:latest

Keepalived

registry.redhat.io/rhceph/keepalived-rhel9:latest

SNMP Gateway

registry.redhat.io/rhceph/snmp-notifier-rhel9:latest

Procedure

  1. Enable the Ceph Ansible repositories on the Ansible administration node:

    Red Hat Enterprise Linux 9

    subscription-manager repos --enable=rhceph-6-tools-for-rhel-9-x86_64-rpms
    Copy to Clipboard Toggle word wrap

  2. Update the cephadm and cephadm-ansible package.

    Example

    [root@admin ~]# dnf update cephadm
[root@admin ~]# dnf update cephadm-ansible
    Copy to Clipboard Toggle word wrap

  3. Run the preflight playbook with the upgrade_ceph_packages parameter set to true and the ceph_origin parameter set to custom on the bootstrapped host in the storage cluster:

    Syntax

    ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true"
    Copy to Clipboard Toggle word wrap

    Example

    [ceph-admin@admin ~]$ ansible-playbook -i /etc/ansible/hosts cephadm-preflight.yml --extra-vars "ceph_origin=custom upgrade_ceph_packages=true"
    Copy to Clipboard Toggle word wrap

    This package upgrades cephadm on all the nodes.

  4. Log into the cephadm shell:

    Example

    [root@node0 ~]# cephadm shell
    Copy to Clipboard Toggle word wrap

  5. Ensure all the hosts are online and that the storage cluster is healthy:

    Example

    [ceph: root@node0 /]# ceph -s
    Copy to Clipboard Toggle word wrap

  6. Set the OSD noout, noscrub, and nodeep-scrub flags to prevent OSDs from getting marked out during upgrade and to avoid unnecessary load on the cluster:

    Example

    [ceph: root@host01 /]# ceph osd set noout
[ceph: root@host01 /]# ceph osd set noscrub
[ceph: root@host01 /]# ceph osd set nodeep-scrub
    Copy to Clipboard Toggle word wrap

  7. Check service versions and the available target containers:

    Syntax

    ceph orch upgrade check IMAGE_NAME
    Copy to Clipboard Toggle word wrap

    Example

    [ceph: root@node0 /]# ceph orch upgrade check LOCAL_NODE_FQDN:5000/rhceph/rhceph-6-rhel9
    Copy to Clipboard Toggle word wrap

  8. Upgrade the storage cluster:

    Syntax

    ceph orch upgrade start IMAGE_NAME
    Copy to Clipboard Toggle word wrap

    Example

    [ceph: root@node0 /]# ceph orch upgrade start LOCAL_NODE_FQDN:5000/rhceph/rhceph-6-rhel9
    Copy to Clipboard Toggle word wrap

    While the upgrade is underway, a progress bar appears in the ceph status output.

    Example

    [ceph: root@node0 /]# ceph status
[...]
progress:
    Upgrade to 17.2.6-70.el9cp (1s)
      [............................]
    Copy to Clipboard Toggle word wrap

  9. Verify the new IMAGE_ID and VERSION of the Ceph cluster:

    Example

    [ceph: root@node0 /]# ceph version
[ceph: root@node0 /]# ceph versions
[ceph: root@node0 /]# ceph orch ps
    Copy to Clipboard Toggle word wrap

  10. When the upgrade is complete, unset the noout, noscrub, and nodeep-scrub flags:

    Example

    [ceph: root@host01 /]# ceph osd unset noout
[ceph: root@host01 /]# ceph osd unset noscrub
[ceph: root@host01 /]# ceph osd unset nodeep-scrub
    Copy to Clipboard Toggle word wrap

Retour au début
Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Theme

© 2025 Red Hat