Appendix I. Ceph scrubbing options
Ceph ensures data integrity by scrubbing placement groups. The following are the Ceph scrubbing options that you can adjust to increase or decrease scrubbing operations.
You can set these configuration options with the ceph config set global CONFIGURATION_OPTION VALUE command.
- mds_max_scrub_ops_in_progress
- Description
-
The maximum number of scrub operations performed in parallel. You can set this value with
ceph config set mds_max_scrub_ops_in_progress VALUEcommand. - Type
- integer
- Default
- 5
- osd_max_scrubs
- Description
- The maximum number of simultaneous scrub operations for a Ceph OSD Daemon.
- Type
- integer
- Default
- 1
- osd_scrub_begin_hour
- Description
-
The specific hour at which the scrubbing begins. Along with
osd_scrub_end_hour, you can define a time window in which the scrubs can happen. Useosd_scrub_begin_hour = 0andosd_scrub_end_hour = 0to allow scrubbing the entire day. - Type
- integer
- Default
- 0
- Allowed range
-
[0, 23]
- osd_scrub_end_hour
- Description
-
The specific hour at which the scrubbing ends. Along with
osd_scrub_begin_hour, you can define a time window, in which the scrubs can happen. Useosd_scrub_begin_hour = 0andosd_scrub_end_hour = 0to allow scrubbing for the entire day. - Type
- integer
- Default
- 0
- Allowed range
-
[0, 23]
- osd_scrub_begin_week_day
- Description
-
The specific day on which the scrubbing begins. 0 = Sunday, 1 = Monday, etc. Along with “osd_scrub_end_week_day”, you can define a time window in which scrubs can happen. Use
osd_scrub_begin_week_day = 0andosd_scrub_end_week_day = 0to allow scrubbing for the entire week. - Type
- integer
- Default
- 0
- Allowed range
-
[0, 6]
- osd_scrub_end_week_day
- Description
-
This defines the day on which the scrubbing ends. 0 = Sunday, 1 = Monday, etc. Along with
osd_scrub_begin_week_day, they define a time window, in which the scrubs can happen. Useosd_scrub_begin_week_day = 0andosd_scrub_end_week_day = 0to allow scrubbing for the entire week. - Type
- integer
- Default
- 0
- Allowed range
-
[0, 6]
- osd_scrub_during_recovery
- Description
-
Allow scrub during recovery. Setting this to
falsedisables scheduling new scrub, and deep-scrub, while there is an active recovery. The already running scrubs continue which is useful to reduce load on busy storage clusters. - Type
- boolean
- Default
- false
- osd_scrub_load_threshold
- Description
- The normalized maximum load. Scrubbing does not happen when the system load, as defined by getloadavg() / number of online CPUs, is higher than this defined number.
- Type
- float
- Default
- 0.5
- osd_scrub_min_interval
- Description
- The minimal interval in seconds for scrubbing the Ceph OSD daemon when the Ceph storage Cluster load is low.
- Type
- float
- Default
- 1 day
- osd_scrub_max_interval
- Description
- The maximum interval in seconds for scrubbing the Ceph OSD daemon irrespective of cluster load.
- Type
- float
- Default
- 7 days
- osd_scrub_chunk_min
- Description
- The minimal number of object store chunks to scrub during a single operation. Ceph blocks writes to a single chunk during scrub.
- type
- integer
- Default
- 5
- osd_scrub_chunk_max
- Description
- The maximum number of object store chunks to scrub during a single operation.
- type
- integer
- Default
- 25
- osd_scrub_sleep
- Description
- Time to sleep before scrubbing the next group of chunks. Increasing this value slows down the overall rate of scrubbing, so that client operations are less impacted.
- type
- float
- Default
- 0.0
- osd_scrub_extended_sleep
- Description
- Duration to inject a delay during scrubbing out of scrubbing hours or seconds.
- type
- float
- Default
- 0.0
- osd_scrub_backoff_ratio
- Description
- Backoff ratio for scheduling scrubs. This is the percentage of ticks that do NOT schedule scrubs, 66% means that 1 out of 3 ticks schedules scrubs.
- type
- float
- Default
- 0.66
- osd_deep_scrub_interval
- Description
-
The interval for
deepscrubbing, fully reading all data. Theosd_scrub_load_thresholddoes not affect this setting. - type
- float
- Default
- 7 days
- osd_debug_deep_scrub_sleep
- Description
- Inject an expensive sleep during deep scrub IO to make it easier to induce preemption.
- type
- float
- Default
- 0
- osd_scrub_interval_randomize_ratio
- Description
-
Add a random delay to
osd_scrub_min_intervalwhen scheduling the next scrub job for a placement group. The delay is a random value less thanosd_scrub_min_interval*osd_scrub_interval_randomized_ratio. The default setting spreads scrubs throughout the allowed time window of[1, 1.5]*osd_scrub_min_interval. - type
- float
- Default
- 0.5
- osd_deep_scrub_stride
- Description
- Read size when doing a deep scrub.
- type
- size
- Default
- 512 KB
- osd_scrub_auto_repair_num_errors
- Description
- Auto repair does not occur if more than these many errors are found.
- type
- integer
- Default
- 5
- osd_scrub_auto_repair
- Description
-
Setting this to
trueenables automatic Placement Group (PG) repair when errors are found by scrubs or deep-scrubs. However, if more thanosd_scrub_auto_repair_num_errorserrors are found, a repair is NOT performed. This option is for periodic scrubs and not for operator-initiated scrubs. - type
- boolean
- Default
- false
- osd_scrub_max_preemptions
- Description
- Set the maximum number of times you need to preempt a deep scrub due to a client operation before blocking client IO to complete the scrub.
- type
- integer
- Default
- 5
- osd_deep_scrub_keys
- Description
- Number of keys to read from an object at a time during deep scrub.
- type
- integer
- Default
- 1024
