Red Hat Summit Red Hat SummitSupportConsoleDéveloppeursCommencer un essaiContact

Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 2. Colocation of Containerized Ceph Daemons

This section describes:

2.1. How colocation works and its advantages
Copier lien

You can colocate containerized Ceph daemons on the same node. Here are the advantages of colocating some of Ceph’s services:

  • Significant improvement in total cost of ownership (TCO) at small scale
  • Reduction from six nodes to three for the minimum configuration
  • Easier upgrade
  • Better resource isolation

How Colocation Works

You can colocate one daemon from the following list with an OSD daemon by adding the same node to appropriate sections in the Ansible inventory file.

  • The Ceph Object Gateway (radosgw)
  • Metadata Server (MDS)
  • RBD mirror (rbd-mirror)
  • Monitor and the Ceph Manager daemon (ceph-mgr)
  • NFS Ganesha

The following example shows how the inventory file with colocated daemons can look like:

Example 2.1. Ansible inventory file with colocated daemons

[mons]
<hostname1>
<hostname2>
<hostname3>

[mgrs]
<hostname1>
<hostname2>
<hostname3>

[osds]
<hostname4>
<hostname5>
<hostname6>

[rgws]
<hostname4>
<hostname5>
Copy to Clipboard Toggle word wrap

The Figure 2.1, “Colocated Daemons” and Figure 2.2, “Non-colocated Daemons” images shows the difference between clusters with colocated and non-colocated daemons.

Figure 2.1. Colocated Daemons

containers colocated daemons
View larger image
containers colocated daemons

Figure 2.2. Non-colocated Daemons

containers non colocated daemons
View larger image
containers non colocated daemons

When you colocate two containerized Ceph daemons on a same node, the ceph-ansible playbook reserves dedicated CPU and RAM resources to each. By default, ceph-ansible uses values listed in the Recommended Minimum Hardware chapter in the Red Hat Ceph Storage Hardware Selection Guide 3. To learn how to change the default values, see the Setting Dedicated Resources for Colocated Daemons section.

2.2. Setting Dedicated Resources for Colocated Daemons
Copier lien

When colocating two Ceph daemon on the same node, the ceph-ansible playbook reserves CPU and RAM resources for each daemon. The default values that ceph-ansible uses are listed in the Recommended Minimum Hardware chapter in the Red Hat Ceph Storage Hardware Selection Guide. To change the default values, set the needed parameters when deploying Ceph daemons.

Procedure

  1. To change the default CPU limit for a daemon, set the ceph_daemon-type_docker_cpu_limit parameter in the appropriate .yml configuration file when deploying the daemon. See the following table for details.

    Expand
    DaemonParameterConfiguration file

    OSD

    ceph_osd_docker_cpu_limit

    osds.yml

    MDS

    ceph_mds_docker_cpu_limit

    mdss.yml

    RGW

    ceph_rgw_docker_cpu_limit

    rgws.yml

    For example, to change the default CPU limit to 2 for the Ceph Object Gateway, edit the /usr/share/ceph-ansible/group_vars/rgws.yml file as follows: 

    ceph_rgw_docker_cpu_limit: 2
    Copy to Clipboard Toggle word wrap

  2. To change the default RAM for OSD daemons, set the osd_memory_target in the /usr/share/ceph-ansible/group_vars/all.yml file when deploying the daemon. For example, to limit the OSD RAM to 6 GB: 

    ceph_conf_overrides:
  osd:
    osd_memory_target=6000000000
    Copy to Clipboard Toggle word wrap
    Important

    In an hyperconverged infrastructure (HCI) configuration, using the osd_memory_target parameter is the recommended way to limit memory of OSDs. The ceph_osd_docker_memory_limit parameter should not be necessary, but if you wish to use it, then set ceph_osd_docker_memory_limit to 50% higher than osd_memory_target, so that the CGroup limit is more constraining than it is by default for an HCI configuration. For example, if osd_memory_target is set to 6 GB, set ceph_osd_docker_memory_limit to 9 GB: 

    ceph_osd_docker_memory_limit: 9g
    Copy to Clipboard Toggle word wrap
Warning

The ceph_osd_docker_memory_limit parameter sets a hard limit. If the value is exceeded, the OSD can stop running if it is used. The osd_memory_target parameter sets a soft limit so the container will not stop running and interrupt service if the value is exceeded.

Additional Resources

  • The sample configuration files in the /usr/share/ceph-ansible/group_vars/ directory

2.3. Additional Resources
Copier lien

Retour au début
Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Theme

© 2025 Red Hat