Red Hat SummitEste conteúdo não está disponível no idioma selecionado.
Chapter 18. Setting Ceph OSD full thresholds
You can set Ceph OSD full thresholds using the ODF CLI tool or by updating the StorageCluster CR.
18.1. Setting Ceph OSD full thresholds using the ODF CLI tool
You can set Ceph OSD full thresholds temporarily by using the ODF CLI tool. This is necessary in cases when the cluster gets into a full state and the thresholds need to be immediately increased.
Prerequisites
- Download the OpenShift Data Foundation command line interface (CLI) tool. With the Data Foundation CLI tool, you can effectively manage and troubleshoot your Data Foundation environment from a terminal. You can find a compatible version and download the CLI tool from the customer portal.
Procedure
Use the
odf setcommand to adjust Ceph full thresholds. Thesetcommand supports the subcommandsfull,backfillfull, andnearfull.NoteEnsure that the thresholds follow the correct order:
nearfull<backfillfull<full. This means the ratio values fornearfullandbackfillfullmust always remain lower than the ratio offull.
The following examples demonstrate how to use each subcommand.
full
This subcommand allows updating the Ceph OSD full ratio in case Ceph prevents the IO operation on OSDs that reached the specified capacity. The default is 0.85.
If the value is set too close to 1.0, the cluster becomes unrecoverable if the OSDs are full and there is nowhere to grow.
For example, set Ceph OSD full ratio to 0.9 and then add capacity:
$ odf set full 0.9For instructions to add capacity for your specific use case, see the Scaling storage guide.
If OSDs continue to be in stuck, pending, or do not come up at all, perform the following steps:
- Stop all IOs.
Increase the
fullratio to0.92:$ odf set full 0.92Wait for the cluster rebalance to happen. Once cluster rebalance is complete, change the
fullratio back to its original value of 0.85:$ odf set full 0.85
backfillfull
This subcommand allows updating the Ceph OSDs backfillfull ratio in case Ceph denies backfilling to the OSD that reached the capacity specified. The default value is 0.80.
If the value is set too close to 1.0, the OSDs become full and the cluster is not able to backfill.
For example, set backfillfull to 0.85:
$ odf set backfillfull 0.85nearfull
This subcommand allows updating the Ceph OSD nearfull ratio in case Ceph returns the nearfull OSDs message when the cluster reaches the capacity specified. The default value is 0.75.
For example, set nearfull to 0.8:
$ odf set nearfull 0.818.2. Setting Ceph OSD full thresholds by updating the StorageCluster CR
You can set Ceph OSD full thresholds by updating the StorageCluster CR. Use this procedure if you want to override the default settings.
Procedure
You can update the StorageCluster CR to change the settings for full, backfillfull, and nearfull.
full
Use this following command to update the Ceph OSD full ratio in case Ceph prevents the IO operation on OSDs that reached the specified capacity. The default is 0.85.
If the value is set too close to 1.0, the cluster becomes unrecoverable if the OSDs are full and there is nowhere to grow.
For example, to set Ceph OSD full ratio to 0.9:
$ oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephCluster/fullRatio", "value": 0.90 }]'backfillfull
Use the following command to set the Ceph OSDd backfillfull ratio in case Ceph denies backfilling to the OSD that reached the capacity specified. The default value is 0.80.
If the value is set too close to 1.0, the OSDs become full and the cluster is not able to backfill.
For example, set backfill full to 0.85:
$ oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephCluster/backfillFullRatio", "value": 0.85 }]'nearfull
Use the following command to set the Ceph OSD nearfull ratio in case Ceph returns the nearfull OSDs message when the cluster reaches the capacity specified. The default value is 0.75.
For example, set nearfull to 0.8:
$ oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/managedResources/cephCluster/nearFullRatio", "value": 0.8 }]'
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}