SupportConsoleDevelopersStart a trialContact

Chapter 1. Deploying an overcloud and Red Hat Ceph Storage

Red Hat OpenStack Platform (RHOSP) director deploys the cloud environment, also known as the overcloud, and Red Hat Ceph Storage. Director uses Ansible playbooks provided through the tripleo-ansible package to deploy the Ceph Storage cluster. The director also manages the configuration and scaling operations of the Ceph Storage cluster.

For more information about Red Hat Ceph Storage, see Red Hat Ceph Storage Architecture Guide.

For more information about services in the Red Hat OpenStack Platform, see Configuring a basic overcloud with the CLI tools in Installing and managing Red Hat OpenStack Platform with director.

1.1. Red Hat Ceph Storage clusters
Copy link

Red Hat Ceph Storage is a distributed data object store designed for performance, reliability, and scalability. Distributed object stores use unstructured data to simultaneously service modern and legacy object interfaces.

Ceph Storage is deployed as a cluster. A Ceph Storage cluster consists of two primary types of daemons:

  • Ceph Object Storage Daemon (CephOSD) - The CephOSD performs data storage, data replication, rebalancing, recovery, monitoring, and reporting tasks.
  • Ceph Monitor (CephMon) - The CephMon maintains the primary copy of the cluster map with the current state of the cluster.

For more information about Red Hat Ceph Storage, see the Red Hat Ceph Storage Architecture Guide.

RHOSP 17.1 is supported on RHEL 9.2. However, you can optionally update nodes that are mapped to the Ceph Storage role to the latest RHEL release.

1.3. Red Hat Ceph Storage compatibility
Copy link

RHOSP 17.1 supports Red Hat Ceph Storage 6 (RHCSv6) for new deployments. For information about the RHEL versions that are compatible with RHCSv6, see the Red Hat Knowledge Base article Red Hat Ceph Storage: Supported configurations.

RHOSP 17.1 only supports RHCSv5 in the following deployment cases: RHOSP 17.1 only supports RHCSv5 in the following deployment cases:

  • You are upgrading from RHOSP 16.2 and RHCSv4.
  • You upgraded from 17.0 with RHCSv5 and have not yet upgraded to RHCSv6.

1.4. Deploying Red Hat Ceph Storage
Copy link

You deploy Red Hat Ceph Storage in two phases:

  • Create the Red Hat Ceph Storage cluster before deploying the overcloud.
  • Configure the Red Hat Ceph Storage cluster during overcloud deployment.

A Ceph Storage cluster is created ready to serve the Ceph RADOS Block Device (RBD) service. Additionally, the following services are running on the appropriate nodes:

  • Ceph Monitor (CephMon)
  • Ceph Manager (CephMgr)
  • Ceph OSD (CephOSD)

Pools and cephx keys are created during the configuration phase.

The following Ceph Storage components are not available until after the configuration phase:

  • Ceph Dashboard (CephDashboard)
  • Ceph Object Gateway (CephRGW)
  • Ceph MDS (CephMds)

Red Hat Ceph Storage cluster configuration finalizes during overcloud deployment. Daemons and services such as Ceph Object Gateway and Ceph Dashboard deploy according to the overcloud definition. Red Hat OpenStack Platform (RHOSP) services are configured as Ceph Storage cluster clients.

1.5. Red Hat Ceph Storage deployment requirements
Copy link

Provisioning of network resources and bare metal instances is required before Ceph Storage cluster creation. Configure the following before creating a Red Hat Ceph Storage cluster:

  • Provision networks with the openstack overcloud network provision command and the cli-overcloud-network-provision.yaml ansible playbook.
  • Provision bare metal instances with the openstack overcloud node provision command to provision bare metal instances using the cli-overcloud-node-provision.yaml ansible playbook.

For more information about these tasks, see:

The following elements must be present in the overcloud environment to finalize the Ceph Storage cluster configuration:

  • Red Hat OpenStack Platform director installed on an undercloud host. See Installing director in Installing and managing Red Hat OpenStack Platform with director.
  • Installation of recommended hardware to support Red Hat Ceph Storage. For more information about recommended hardware, see the Red Hat Ceph Storage Hardware Guide.

1.5.1. Configuring jumbo frames
Copy link

Use the network definition file to modify the maximum transmission unit (MTU) of your storage interfaces to 9000.

Jumbo frames are frames with an MTU of 9,000. Jumbo frames are not mandatory for the Storage and Storage Management networks but the increase in MTU size improves storage performance. If you want to use jumbo frames, you must configure all network switch ports in the data path to support jumbo frames.

Important

Network configuration changes such as MTU settings must be completed during the initial deployment. They cannot be applied to an existing deployment.

Procedure

  1. Log in to the undercloud node as the stack user.
  2. Locate the network definition file.

  3. Modify the network definition file to extend the template to include the StorageMgmtIpSubnet and StorageMgmtNetworkVlanID attributes of the Storage Management network. Set the mtu attribute of the interfaces to 9000.

    The following is an example of implementing these interface settings: 

    -
    type: interface
    name: em2
    use_dhcp: false
    mtu: 9000
-
    type: vlan
    device: em2
    mtu: 9000
    use_dhcp: false
    vlan_id: {get_param: StorageMgmtNetworkVlanID}
    addresses:
    -
        ip_netmask: {get_param: StorageMgmtIpSubnet}
-
    type: vlan
    device: em2
    mtu: 9000
    use_dhcp: false
    vlan_id: {get_param: StorageNetworkVlanID}
    addresses:
    -
        ip_netmask: {get_param: StorageIpSubnet}
    Note

    For a complete example of a network definition file, see Example network definition file.

  4. Save the changes to the network definition file.

    Note

    All network switch ports between servers using the interface with the new MTU setting must be updated to support jumbo frames. If these switch changes are not made, problems will develop at the application layer that can cause the Red Hat Ceph Storage cluster to not reach quorum. If these settings are made and these problems are still observed, verify all hosts using the network configured for jumbo frames can communicate at the configured MTU setting. Use a command like the following example to perform this task:

    ping -M do -s 8972 172.16.1.11

1.6. Post deployment verification
Copy link

Director deploys a Ceph Storage cluster ready to serve Ceph RADOS Block Device (RBD) using tripleo-ansible roles executed by the cephadm command.

Verify the following are in place after cephadm completes Ceph Storage deployment:

  • SSH access to a CephMon service node to use the sudo cephadm shell command.

  • All OSDs operational.

    Note

    Check inoperative OSDs for environmental issues like uncleaned disks.

  • A Ceph configuration file and client administration keyring file in the /etc/ceph directory of CephMon service nodes.
  • The Ceph Storage cluster is ready to serve RBD.

Pools, cephx keys, CephDashboard, and CephRGW are configured during overcloud deployment by the openstack overcloud deploy command. This is for two reasons:

  • The Dashboard and RGW services must integrate with haproxy. This is deployed with the overcloud.
  • The creation of pools and cephx keys are dependent on which OpenStack clients are deployed.

These resources are created in the Ceph Storage cluster using the client administration keyring file and the ~/deployed_ceph.yaml file output by the openstack overcloud ceph deploy command.

For more information about cephadm, see Red Hat Ceph Storage Installation Guide.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top