Chapter 18. Setting Ceph OSD full thresholds

Prerequisites

• Download the OpenShift Data Foundation command line interface (CLI) tool. With the Data Foundation CLI tool, you can effectively manage and troubleshoot your Data Foundation environment from a terminal. You can find a compatible version and download the CLI tool from the customer portal.

Procedure

• Use the odf set command to adjust Ceph full thresholds. The set command supports the subcommands full, backfillfull, and nearfull.

  Note

  Ensure that the ratio of nearfull and backfill is less than the ratio of full. That means, the value of full should not exceed the value of backfill and the value of backfill should not exceed the value of nearful.

  See the following examples for how to use each subcommand.

  full

  This subcommand allows updating the Ceph OSD full ratio in case Ceph prevents the IO operation on OSDs that reached the specified capacity. The default is 0.85.

  Note

  If the value is set too close to 1.0, the cluster becomes unrecoverable if the OSDs are full and there is nowhere to grow.

  For example, set Ceph OSD full ratio to 0.9 and then add capacity:

  $ odf set full 0.9

  Copy to Clipboard Toggle word wrap

  For instructions to add capacity for your specific use case, see the Scaling storage guide.

  If OSDs continue to be in stuck, pending, or do not come up at all, perform the following steps:

  1. Stop all IOs.

  2. Increase the full ratio to 0.92:

     $ odf set full 0.92

     Copy to Clipboard Toggle word wrap

  3. Wait for the cluster rebalance to happen. Once cluster rebalance is complete, change the full ratio back to its original value of 0.85:

     $ odf set full 0.85

     Copy to Clipboard Toggle word wrap

     backfillfull

     This subcommand allows updating the Ceph OSDs backfillfull ratio in case Ceph denies backfilling to the OSD that reached the capacity specified. The default value is 0.80.

     Note

     If the value is set too close to 1.0, the OSDs become full and the cluster is not able to backfill.

     For example, to set backfillfull to 0.85:

     $ odf set backfillfull 0.85

     Copy to Clipboard Toggle word wrap

     nearfull

     This subcommand allows updating the Ceph OSD nearfull ratio in case Ceph returns the nearfull OSDs message when the cluster reaches the capacity specified. The default value is 0.75.

     For example, to set nearfull to 0.8:

     $ odf set nearfull 0.8

     Copy to Clipboard Toggle word wrap