Appendix C. Ceph Monitor configuration options

mon_initial_members

The IDs of initial monitors in a cluster during startup. If specified, Ceph requires an odd number of monitors to form an initial quorum (for example, 3).

String

None

mon_force_quorum_join

Force monitor to join quorum even if it has been previously removed from the map

Boolean

False

mon_dns_srv_name

The service name used for querying the DNS for the monitor hosts/addresses.

String

ceph-mon

fsid

The cluster ID. One per cluster.

UUID

N/A. May be generated by a deployment tool if not specified.

mon_data

The monitor’s data location.

String

/var/lib/ceph/mon/$cluster-$id

mon_data_size_warn

Ceph issues a HEALTH_WARN status in the cluster log when the monitor’s data store reaches this threshold. The default value is 15GB.

Integer

15*1024*1024*1024*

mon_data_avail_warn

Ceph issues a HEALTH_WARN status in the cluster log when the available disk space of the monitor’s data store is lower than or equal to this percentage.

Integer

30

mon_data_avail_crit

Ceph issues a HEALTH_ERR status in the cluster log when the available disk space of the monitor’s data store is lower or equal to this percentage.

Integer

5

mon_warn_on_cache_pools_without_hit_sets

Ceph issues a HEALTH_WARN status in the cluster log if a cache pool does not have the hit_set_type parameter set.

Boolean

True

mon_warn_on_crush_straw_calc_version_zero

Ceph issues a HEALTH_WARN status in the cluster log if the CRUSH’s straw_calc_version is zero. See CRUSH tunables for details.

Boolean

True

mon_warn_on_legacy_crush_tunables

Ceph issues a HEALTH_WARN status in the cluster log if CRUSH tunables are too old (older than mon_min_crush_required_version).

Boolean

True

mon_crush_min_required_version

This setting defines the minimum tunable profile version required by the cluster.

String

hammer

mon_warn_on_osd_down_out_interval_zero

Ceph issues a HEALTH_WARN status in the cluster log if the mon_osd_down_out_interval setting is zero, because the Leader behaves in a similar manner when the noout flag is set. Administrators find it easier to troubleshoot a cluster by setting the noout flag. Ceph issues the warning to ensure administrators know that the setting is zero.

Boolean

True

mon_cache_target_full_warn_ratio

Ceph issues a warning when between the ratio of cache_target_full and target_max_object.

Float

0.66

mon_health_data_update_interval

How often (in seconds) a monitor in the quorum shares its health status with its peers. A negative number disables health updates.

Float

60

mon_health_to_clog

This setting enables Ceph to send a health summary to the cluster log periodically.

Boolean

True

mon_health_detail_to_clog

This setting enable Ceph to send a health details to the cluster log periodically.

Boolean

True

mon_op_complaint_time

Number of seconds after which the Ceph Monitor operation is considered blocked after no updates.

Integer

30

mon_health_to_clog_tick_interval

How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. If the current health summary is empty or identical to the last time, the monitor will not send the status to the cluster log.

Float

60.000000

mon_health_to_clog_interval

How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. The monitor will always send the summary to the cluster log.

Integer

600

mon_osd_full_ratio

The percentage of disk space used before an OSD is considered full.

Float

.95

mon_osd_nearfull_ratio

The percentage of disk space used before an OSD is considered nearfull.

Float

.85

mon_sync_trim_timeout

No description available.

Double

30.0

mon_sync_heartbeat_timeout

No description available.

Double

30.0

mon_sync_heartbeat_interval

No description available.

Double

5.0

mon_sync_backoff_timeout

No description available.

Double

30.0

mon_sync_timeout

The number of seconds the monitor will wait for the next update message from its sync provider before it gives up and bootstraps again.

Double

60.000000

mon_sync_max_retries

Integer

5

mon_sync_max_payload_size

The maximum size for a sync payload (in bytes).

32-bit Integer

1045676

paxos_max_join_drift

The maximum Paxos iterations before we must first sync the monitor data stores. When a monitor finds that its peer is too far ahead of it, it will first sync with data stores before moving on.

Integer

10

paxos_stash_full_interval

How often (in commits) to stash a full copy of the PaxosService state. Currently this setting only affects mds, mon, auth, and mgr PaxosServices.

Integer

25

paxos_propose_interval

Gather updates for this time interval before proposing a map update.

Double

1.0

paxos_min

The minimum number of paxos states to keep around

Integer

500

paxos_min_wait

The minimum amount of time to gather updates after a period of inactivity.

Double

0.05

paxos_trim_min

Number of extra proposals tolerated before trimming

Integer

250

paxos_trim_max

The maximum number of extra proposals to trim at a time

Integer

500

paxos_service_trim_min

The minimum amount of versions to trigger a trim (0 disables it)

Integer

250

paxos_service_trim_max

The maximum amount of versions to trim during a single proposal (0 disables it)

Integer

500

mon_max_log_epochs

The maximum amount of log epochs to trim during a single proposal

Integer

500

mon_max_pgmap_epochs

The maximum amount of pgmap epochs to trim during a single proposal

Integer

500

mon_mds_force_trim_to

Force monitor to trim mdsmaps to this point (0 disables it. Dangerous, use with care)

Integer

0

mon_osd_force_trim_to

Force monitor to trim osdmaps to this point, even if there is PGs not clean at the specified epoch (0 disables it. dangerous, use with care)

Integer

0

mon_osd_cache_size

The size of osdmaps cache, not to rely on underlying store’s cache

Integer

500

mon_election_timeout

On election proposer, maximum waiting time for all ACKs in seconds.

Float

5

mon_lease

The length (in seconds) of the lease on the monitor’s versions.

Float

5

mon_lease_renew_interval_factor

mon lease * mon lease renew interval factor will be the interval for the Leader to renew the other monitor’s leases. The factor should be less than 1.0.

Float

0.6

mon_lease_ack_timeout_factor

The Leader will wait mon lease * mon lease ack timeout factor for the Providers to acknowledge the lease extension.

Float

2.0

mon_min_osdmap_epochs

Minimum number of OSD map epochs to keep at all times.

32-bit Integer

500

mon_max_pgmap_epochs

Maximum number of PG map epochs the monitor should keep.

32-bit Integer

500

mon_max_log_epochs

Maximum number of Log epochs the monitor should keep.

32-bit Integer

500

clock_offset

How much to offset the system clock. See Clock.cc for details.

Double

0

mon_tick_interval

A monitor’s tick interval in seconds.

32-bit Integer

5

mon_clock_drift_allowed

The clock drift in seconds allowed between monitors.

Float

.050

mon_clock_drift_warn_backoff

Exponential backoff for clock drift warnings.

Float

5

mon_timecheck_interval

The time check interval (clock drift check) in seconds for the leader.

Float

300.0

mon_timecheck_skew_interval

The time check interval (clock drift check) in seconds when in the presence of a skew in seconds for the Leader.

Float

30.0

mon_max_osd

The maximum number of OSDs allowed in the cluster.

32-bit Integer

10000

mon_globalid_prealloc

The number of global IDs to pre-allocate for clients and daemons in the cluster.

32-bit Integer

10000

mon_sync_fs_threshold

Synchronize with the filesystem when writing the specified number of objects. Set it to 0 to disable it.

32-bit Integer

5

mon_subscribe_interval

The refresh interval, in seconds, for subscriptions. The subscription mechanism enables obtaining the cluster maps and log information.

Double

86400.000000

mon_stat_smooth_intervals

Ceph will smooth statistics over the last N PG maps.

Integer

6

mon_probe_timeout

Number of seconds the monitor will wait to find peers before bootstrapping.

Double

2.0

mon_daemon_bytes

The message memory cap for metadata server and OSD messages (in bytes).

64-bit Integer Unsigned

400ul << 20

mon_max_log_entries_per_event

The maximum number of log entries per event.

Integer

4096

mon_osd_prime_pg_temp

Enables or disable priming the PGMap with the previous OSDs when an out OSD comes back into the cluster. With the true setting, the clients will continue to use the previous OSDs until the newly in OSDs as that PG peered.

Boolean

true

mon_osd_prime_pg_temp_max_time

How much time in seconds the monitor should spend trying to prime the PGMap when an out OSD comes back into the cluster.

Float

0.5

mon_lease_ack_timeout_factor

The Leader will wait mon lease * mon lease ack timeout factor for the Providers to acknowledge the lease extension.

Float

2.0

mon_accept_timeout_factor

The Leader will wait mon lease * mon accept timeout factor for the Requesters to accept a Paxos update. It is also used during the Paxos recovery phase for similar purposes.

Float

2.0

mon_min_osdmap_epochs

Minimum number of OSD map epochs to keep at all times.

32-bit Integer

500

mon_max_pgmap_epochs

Maximum number of PG map epochs the monitor should keep.

32-bit Integer

500

mon_max_log_epochs

Maximum number of Log epochs the monitor should keep.

32-bit Integer

500

clock_offset

How much to offset the system clock. See Clock.cc for details.

Double

0

mon_tick_interval

A monitor’s tick interval in seconds.

32-bit Integer

5

mon_clock_drift_allowed

The clock drift in seconds allowed between monitors.

Float

.050

mon_clock_drift_warn_backoff

Exponential backoff for clock drift warnings.

Float

5

mon_timecheck_interval

The time check interval (clock drift check) in seconds for the leader.

Float

300.0

mon_timecheck_skew_interval

The time check interval (clock drift check) in seconds when in the presence of a skew in seconds for the Leader.

Float

30.0

mon_max_osd

The maximum number of OSDs allowed in the cluster.

32-bit Integer

10000

mon_globalid_prealloc

The number of global IDs to pre-allocate for clients and daemons in the cluster.

32-bit Integer

10000

mon_sync_fs_threshold

Synchronize with the filesystem when writing the specified number of objects. Set it to 0 to disable it.

32-bit Integer

5

mon_subscribe_interval

The refresh interval, in seconds, for subscriptions. The subscription mechanism enables obtaining the cluster maps and log information.

Double

86400.000000

mon_stat_smooth_intervals

Ceph will smooth statistics over the last N PG maps.

Integer

6

mon_probe_timeout

Number of seconds the monitor will wait to find peers before bootstrapping.

Double

2.0

mon_daemon_bytes

The message memory cap for metadata server and OSD messages (in bytes).

64-bit Integer Unsigned

400ul << 20

mon_max_log_entries_per_event

The maximum number of log entries per event.

Integer

4096

mon_osd_prime_pg_temp

Enables or disable priming the PGMap with the previous OSDs when an out OSD comes back into the cluster. With the true setting, the clients will continue to use the previous OSDs until the newly in OSDs as that PG peered.

Boolean

true

mon_osd_prime_pg_temp_max_time

How much time in seconds the monitor should spend trying to prime the PGMap when an out OSD comes back into the cluster.

Float

0.5

mon_osd_prime_pg_temp_max_time_estimate

Maximum estimate of time spent on each PG before we prime all PGs in parallel.

Float

0.25

mon_osd_allow_primary_affinity

Allow primary_affinity to be set in the osdmap.

Boolean

False

mon_osd_pool_ec_fast_read

Whether turn on fast read on the pool or not. It will be used as the default setting of newly created erasure pools if fast_read is not specified at create time.

Boolean

False

mon_mds_skip_sanity

Skip safety assertions on FSMap, in case of bugs where we want to continue anyway. Monitor terminates if the FSMap sanity check fails, but we can disable it by enabling this option.

Boolean

False

mon_max_mdsmap_epochs

The maximum amount of mdsmap epochs to trim during a single proposal.

Integer

500

mon_config_key_max_entry_size

The maximum size of config-key entry (in bytes).

Integer

65536

mon_warn_pg_not_scrubbed_ratio

The percentage of the scrub max interval past the scrub max interval to warn.

float

0.5

mon_warn_pg_not_deep_scrubbed_ratio

The percentage of the deep scrub interval past the deep scrub interval to warn

float

0.75

mon_scrub_interval

How often, in seconds, the monitor scrub its store by comparing the stored checksums with the computed ones of all the stored keys.

Integer

3600*24

mon_scrub_timeout

The timeout to restart scrub of mon quorum participant does not respond for the latest chunk.

Integer

5 min

mon_scrub_max_keys

The maximum number of keys to scrub each time.

Integer

100

mon_scrub_inject_crc_mismatch

The probability of injecting CRC mismatches into Ceph Monitor scrub.

Integer

3600*24

mon_scrub_inject_missing_keys

The probability of injecting missing keys into mon scrub.

Float

0

mon_compact_on_start

Compact the database used as Ceph Monitor store on ceph-mon start. A manual compaction helps to shrink the monitor database and improve its performance if the regular compaction fails to work.

Boolean

False

mon_compact_on_bootstrap

Compact the database used as Ceph Monitor store on bootstrap. The monitor starts probing each other for creating a quorum after bootstrap. If it times out before joining the quorum, it will start over and bootstrap itself again.

Boolean

False

mon_compact_on_trim

Compact a certain prefix (including paxos) when we trim its old states.

Boolean

True

mon_cpu_threads

Number of threads for performing CPU intensive work on monitor.

Integer

4

mon_osd_mapping_pgs_per_chunk

We calculate the mapping from the placement group to OSDs in chunks. This option specifies the number of placement groups per chunk.

Integer

4096

mon_osd_max_split_count

Largest number of PGs per "involved" OSD to let split create. When we increase the pg_num of a pool, the placement groups will be split on all OSDs serving that pool. We want to avoid extreme multipliers on PG splits.

Integer

300

rados_mon_op_timeout

Number of seconds to wait for a response from the monitor before returning an error from a rados operation. 0 means at limit, or no wait time.

Double

0