Appendix C. Ceph Monitor configuration options
The following are Ceph monitor configuration options that can be set up during deployment.
mon_initial_members- Description
- The IDs of initial monitors in a cluster during startup. If specified, Ceph requires an odd number of monitors to form an initial quorum (for example, 3).
- Type
- String
- Default
- None
mon_force_quorum_join- Description
- Force monitor to join quorum even if it has been previously removed from the map
- Type
- Boolean
- Default
-
False
mon_dns_srv_name- Description
- The service name used for querying the DNS for the monitor hosts/addresses.
- Type
- String
- Default
-
ceph-mon
fsid- Description
- The cluster ID. One per cluster.
- Type
- UUID
- Required
- Yes.
- Default
- N/A. May be generated by a deployment tool if not specified.
mon_data- Description
- The monitor’s data location.
- Type
- String
- Default
-
/var/lib/ceph/mon/$cluster-$id
mon_data_size_warn- Description
-
Ceph issues a
HEALTH_WARNstatus in the cluster log when the monitor’s data store reaches this threshold. The default value is 15GB. - Type
- Integer
- Default
-
15*1024*1024*1024*`
mon_data_avail_warn- Description
-
Ceph issues a
HEALTH_WARNstatus in cluster log when the available disk space of the monitor’s data store is lower than or equal to this percentage. - Type
- Integer
- Default
-
30
mon_data_avail_crit- Description
-
Ceph issues a
HEALTH_ERRstatus in cluster log when the available disk space of the monitor’s data store is lower or equal to this percentage. - Type
- Integer
- Default
-
5
mon_warn_on_cache_pools_without_hit_sets- Description
-
Ceph issues a
HEALTH_WARNstatus in cluster log if a cache pool does not have thehit_set_typeparamater set. - Type
- Boolean
- Default
- True
mon_warn_on_crush_straw_calc_version_zero- Description
-
Ceph issues a
HEALTH_WARNstatus in the cluster log if the CRUSH’sstraw_calc_versionis zero. See CRUSH tunables for details. - Type
- Boolean
- Default
- True
mon_warn_on_legacy_crush_tunables- Description
-
Ceph issues a
HEALTH_WARNstatus in the cluster log if CRUSH tunables are too old (older thanmon_min_crush_required_version). - Type
- Boolean
- Default
- True
mon_crush_min_required_version- Description
- This setting defines the minimum tunable profile version required by the cluster.
- Type
- String
- Default
-
firefly
mon_warn_on_osd_down_out_interval_zero- Description
-
Ceph issues a
HEALTH_WARNstatus in the cluster log if themon_osd_down_out_intervalsetting is zero, because the Leader behaves in a similar manner when thenooutflag is set. Administrators find it easier to troubleshoot a cluster by setting thenooutflag. Ceph issues the warning to ensure administrators know that the setting is zero. - Type
- Boolean
- Default
- True
mon_cache_target_full_warn_ratio- Description
-
Ceph issues a warning when between the ratio of
cache_target_fullandtarget_max_object. - Type
- Float
- Default
-
0.66
mon_health_data_update_interval- Description
- How often (in seconds) a monitor in the quorum shares its health status with its peers. A negative number disables health updates.
- Type
- Float
- Default
-
60
mon_health_to_clog- Description
- This setting enable Ceph to send a health summary to the cluster log periodically.
- Type
- Boolean
- Default
- True
mon_health_detail_to_clog- Description
- This setting enable Ceph to send a health details to the cluster log periodically.
- Type
- Boolean
- Default
- True
mon_op_complaint_time- Description
- Number of seconds after which the Ceph Monitor operation is considered blocked after no updates.
- Type
- Integer
- Default
- 30
mon_health_to_clog_tick_interval- Description
- How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. If the current health summary is empty or identical to the last time, the monitor will not send the status to the cluster log.
- Type
- Integer
- Default
- 3600
mon_health_to_clog_interval- Description
- How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. The monitor will always send the summary to cluster log.
- Type
- Integer
- Default
- 60
mon_osd_full_ratio- Description
-
The percentage of disk space used before an OSD is considered
full. - Type
- Float:
- Default
-
.95
mon_osd_nearfull_ratio- Description
-
The percentage of disk space used before an OSD is considered
nearfull. - Type
- Float
- Default
-
.85
mon_sync_trim_timeout- Description, Type
- Double
- Default
-
30.0
mon_sync_heartbeat_timeout- Description, Type
- Double
- Default
-
30.0
mon_sync_heartbeat_interval- Description, Type
- Double
- Default
-
5.0
mon_sync_backoff_timeout- Description, Type
- Double
- Default
-
30.0
mon_sync_timeout- Description
- Number of seconds the monitor will wait for the next update message from its sync provider before it gives up and bootstraps again.
- Type
- Double
- Default
-
30.0
mon_sync_max_retries- Description, Type
- Integer
- Default
-
5
mon_sync_max_payload_size- Description
- The maximum size for a sync payload (in bytes).
- Type
- 32-bit Integer
- Default
-
1045676
paxos_max_join_drift- Description
- The maximum Paxos iterations before we must first sync the monitor data stores. When a monitor finds that its peer is too far ahead of it, it will first sync with data stores before moving on.
- Type
- Integer
- Default
-
10
paxos_stash_full_interval- Description
-
How often (in commits) to stash a full copy of the PaxosService state. Current this setting only affects
mds,mon,authandmgrPaxosServices. - Type
- Integer
- Default
- 25
paxos_propose_interval- Description
- Gather updates for this time interval before proposing a map update.
- Type
- Double
- Default
-
1.0
paxos_min- Description
- The minimum number of paxos states to keep around
- Type
- Integer
- Default
- 500
paxos_min_wait- Description
- The minimum amount of time to gather updates after a period of inactivity.
- Type
- Double
- Default
-
0.05
paxos_trim_min- Description
- Number of extra proposals tolerated before trimming
- Type
- Integer
- Default
- 250
paxos_trim_max- Description
- The maximum number of extra proposals to trim at a time
- Type
- Integer
- Default
- 500
paxos_service_trim_min- Description
- The minimum amount of versions to trigger a trim (0 disables it)
- Type
- Integer
- Default
- 250
paxos_service_trim_max- Description
- The maximum amount of versions to trim during a single proposal (0 disables it)
- Type
- Integer
- Default
- 500
mon_max_log_epochs- Description
- The maximum amount of log epochs to trim during a single proposal
- Type
- Integer
- Default
- 500
mon_max_pgmap_epochs- Description
- The maximum amount of pgmap epochs to trim during a single proposal
- Type
- Integer
- Default
- 500
mon_mds_force_trim_to- Description
- Force monitor to trim mdsmaps to this point (0 disables it. dangerous, use with care)
- Type
- Integer
- Default
- 0
mon_osd_force_trim_to- Description
- Force monitor to trim osdmaps to this point, even if there is PGs not clean at the specified epoch (0 disables it. dangerous, use with care)
- Type
- Integer
- Default
- 0
mon_osd_cache_size- Description
- The size of osdmaps cache, not to rely on underlying store’s cache
- Type
- Integer
- Default
- 10
mon_election_timeout- Description
- On election proposer, maximum waiting time for all ACKs in seconds.
- Type
- Float
- Default
-
5
mon_lease- Description
- The length (in seconds) of the lease on the monitor’s versions.
- Type
- Float
- Default
-
5
mon_lease_renew_interval_factor- Description
-
mon lease*mon lease renew interval factorwill be the interval for the Leader to renew the other monitor’s leases. The factor should be less than1.0. - Type
- Float
- Default
-
0.6
mon_lease_ack_timeout_factor- Description
-
The Leader will wait
mon lease*mon lease ack timeout factorfor the Providers to acknowledge the lease extension. - Type
- Float
- Default
-
2.0
mon_accept_timeout_factor- Description
-
The Leader will wait
mon lease*mon accept timeout factorfor the Requesters to accept a Paxos update. It is also used during the Paxos recovery phase for similar purposes. - Type
- Float
- Default
-
2.0
mon_min_osdmap_epochs- Description
- Minimum number of OSD map epochs to keep at all times.
- Type
- 32-bit Integer
- Default
-
500
mon_max_pgmap_epochs- Description
- Maximum number of PG map epochs the monitor should keep.
- Type
- 32-bit Integer
- Default
-
500
mon_max_log_epochs- Description
- Maximum number of Log epochs the monitor should keep.
- Type
- 32-bit Integer
- Default
-
500
clock_offset- Description
-
How much to offset the system clock. See
Clock.ccfor details. - Type
- Double
- Default
-
0
mon_tick_interval- Description
- A monitor’s tick interval in seconds.
- Type
- 32-bit Integer
- Default
-
5
mon_clock_drift_allowed- Description
- The clock drift in seconds allowed between monitors.
- Type
- Float
- Default
-
.050
mon_clock_drift_warn_backoff- Description
- Exponential backoff for clock drift warnings.
- Type
- Float
- Default
-
5
mon_timecheck_interval- Description
- The time check interval (clock drift check) in seconds for the leader.
- Type
- Float
- Default
-
300.0
mon_timecheck_skew_interval- Description
- The time check interval (clock drift check) in seconds when in the presence of a skew in seconds for the Leader.
- Type
- Float
- Default
-
30.0
mon_max_osd- Description
- The maximum number of OSDs allowed in the cluster.
- Type
- 32-bit Integer
- Default
-
10000
mon_globalid_prealloc- Description
- The number of global IDs to pre-allocate for clients and daemons in the cluster.
- Type
- 32-bit Integer
- Default
-
100
mon_sync_fs_threshold- Description
-
Synchronize with the filesystem when writing the specified number of objects. Set it to
0to disable it. - Type
- 32-bit Integer
- Default
-
5
mon_subscribe_interval- Description
- The refresh interval, in seconds, for subscriptions. The subscription mechanism enables obtaining the cluster maps and log information.
- Type
- Double
- Default
-
300
mon_stat_smooth_intervals- Description
-
Ceph will smooth statistics over the last
NPG maps. - Type
- Integer
- Default
-
2
mon_probe_timeout- Description
- Number of seconds the monitor will wait to find peers before bootstrapping.
- Type
- Double
- Default
-
2.0
mon_daemon_bytes- Description
- The message memory cap for metadata server and OSD messages (in bytes).
- Type
- 64-bit Integer Unsigned
- Default
-
400ul << 20
mon_max_log_entries_per_event- Description
- The maximum number of log entries per event.
- Type
- Integer
- Default
-
4096
mon_osd_prime_pg_temp- Description
-
Enables or disable priming the PGMap with the previous OSDs when an out OSD comes back into the cluster. With the
truesetting the clients will continue to use the previous OSDs until the newly in OSDs as that PG peered. - Type
- Boolean
- Default
-
true
mon_osd_prime_pg_temp_max_time- Description
- How much time in seconds the monitor should spend trying to prime the PGMap when an out OSD comes back into the cluster.
- Type
- Float
- Default
-
0.5
mon_osd_prime_pg_temp_max_time_estimate- Description
- Maximum estimate of time spent on each PG before we prime all PGs in parallel.
- Type
- Float
- Default
-
0.25
mon_osd_allow_primary_affinity- Description
-
allow
primary_affinityto be set in the osdmap. - Type
- Boolean
- Default
- False
mon_osd_pool_ec_fast_read- Description
-
Whether turn on fast read on the pool or not. It will be used as the default setting of newly created erasure pools if
fast_readis not specified at create time. - Type
- Boolean
- Default
- False
mon_mds_skip_sanity- Description
- Skip safety assertions on FSMap, in case of bugs where we want to continue anyway. Monitor terminates if the FSMap sanity check fails, but we can disable it by enabling this option.
- Type
- Boolean
- Default
- False
mon_max_mdsmap_epochs- Description
- The maximum amount of mdsmap epochs to trim during a single proposal.
- Type
- Integer
- Default
- 500
mon_config_key_max_entry_size- Description
- The maximum size of config-key entry (in bytes).
- Type
- Integer
- Default
- 4096
mon_scrub_interval- Description
- How often, in seconds, the monitor scrub its store by comparing the stored checksums with the computed ones of all the stored keys.
- Type
- Integer
- Default
- 3600*24
mon_scrub_max_keys- Description
- The maximum number of keys to scrub each time.
- Type
- Integer
- Default
- 100
mon_compact_on_start- Description
-
Compact the database used as Ceph Monitor store on
ceph-monstart. A manual compaction helps to shrink the monitor database and improve the performance of it if the regular compaction fails to work. - Type
- Boolean
- Default
- False
mon_compact_on_bootstrap- Description
- Compact the database used as Ceph Monitor store on on bootstrap. Monitor starts probing each other for creating a quorum after bootstrap. If it times out before joining the quorum, it will start over and bootstrap itself again.
- Type
- Boolean
- Default
- False
mon_compact_on_trim- Description
- Compact a certain prefix (including paxos) when we trim its old states.
- Type
- Boolean
- Default
- True
mon_cpu_threads- Description
- Number of threads for performing CPU intensive work on monitor.
- Type
- Boolean
- Default
- True
mon_osd_mapping_pgs_per_chunk- Description
- We calculate the mapping from placement group to OSDs in chunks. This option specifies the number of placement groups per chunk.
- Type
- Integer
- Default
- 4096
mon_osd_max_split_count- Description
-
Largest number of PGs per "involved" OSD to let split create. When we increase the
pg_numof a pool, the placement groups will be splitted on all OSDs serving that pool. We want to avoid extreme multipliers on PG splits. - Type
- Integer
- Default
- 300
rados_mon_op_timeout- Description
- Number of seconds to wait for a response from the monitor before returning an error from a rados operation. 0 means at limit, or no wait time.
- Type
- Double
- Default
- 0
Additional Resources
