Chapter 10. CRUSH Rules
CRUSH rules define how a Ceph client selects buckets and the primary OSD within them to store object, and how the primary OSD selects buckets and the secondary OSDs to store replicas (or coding chunks). For example, you might create a rule that selects a pair of target OSDs backed by SSDs for 2 object replicas, and another rule that select 3 target OSDs backed by SAS drives in different data centers for 3 replicas.
A rule takes the following form:
rule <rulename> {
ruleset <ruleset>
type [ replicated | raid4 ]
min_size <min-size>
max_size <max-size>
step take <bucket-type>
step [choose|chooseleaf] [firstn|indep] <N> <bucket-type>
step emit
}ruleset
- Description
- A means of classifying a rule as belonging to a set of rules. Activated by setting the ruleset in a pool.
- Purpose
- A component of the rule mask.
- Type
- Integer
- Required
- Yes
- Default
- 0
type
- Description
- Describes a rule for either a storage drive (replicated) or a RAID.
- Purpose
- A component of the rule mask.
- Type
- String
- Required
- Yes
- Default
-
replicated - Valid Values
-
Currently only
replicated
min_size
- Description
- If a pool makes fewer replicas than this number, CRUSH will NOT select this rule.
- Type
- Integer
- Purpose
- A component of the rule mask.
- Required
- Yes
- Default
-
1
max_size
- Description
- If a pool makes more replicas than this number, CRUSH will NOT select this rule.
- Type
- Integer
- Purpose
- A component of the rule mask.
- Required
- Yes
- Default
- 10
step take <bucket-name>
- Description
- Takes a bucket name, and begins iterating down the tree.
- Purpose
- A component of the rule.
- Required
- Yes
- Example
-
step take data
step choose firstn <num> type <bucket-type>
- Description
Selects the number of buckets of the given type. The number is usually the number of replicas in the pool (i.e., pool size). +
-
If
<num> == 0, choosepool-num-replicasbuckets (all available). -
If
<num> > 0 && < pool-num-replicas, choose that many buckets. -
If
<num> < 0, it meanspool-num-replicas - {num}.
-
If
- Purpose
- A component of the rule.
- Prerequisite
-
Follows
step takeorstep choose. - Example
-
step choose firstn 1 type row
step chooseleaf firstn <num> type <bucket-type>
- Description
Selects a set of buckets of
{bucket-type}and chooses a leaf node from the subtree of each bucket in the set of buckets. The number of buckets in the set is usually the number of replicas in the pool (i.e., pool size). +-
If
<num> == 0, choosepool-num-replicasbuckets (all available). -
If
<num> > 0 && < pool-num-replicas, choose that many buckets. -
If
<num> < 0, it meanspool-num-replicas - <num>.
-
If
- Purpose
- A component of the rule. Usage removes the need to select a device using two steps.
- Prerequisite
-
Follows
step takeorstep choose. - Example
-
step chooseleaf firstn 0 type row
step emit
- Description
- Outputs the current value and empties the stack. Typically used at the end of a rule, but may also be used to pick from different trees in the same rule.
- Purpose
- A component of the rule.
- Prerequisite
-
Follows
step choose. - Example
-
step emit
To activate one or more rules with a common ruleset number to a pool, set the ruleset number of the pool.
10.1. List Rules
To list CRUSH rules from the command line, execute the following:
ceph osd crush rule list ceph osd crush rule ls
10.2. Dump a Rule
To dump the contents of a specific CRUSH rule, execute the following:
ceph osd crush rule dump {name}10.3. Add a Simple Rule
To add a CRUSH rule, you must specify a rule name, the root node of the hierarchy you wish to use, the type of bucket you want to replicate across (e.g., rack, row, etc) and the mode for choosing the bucket.
ceph osd crush rule create-simple {rulename} {root} {bucket-type} {first|indep}
Ceph will create a rule with chooseleaf and 1 bucket of the type you specify.
For example:
ceph osd crush rule create-simple deleteme default host firstn
Create the following rule:
{ "rule_id": 1,
"rule_name": "deleteme",
"ruleset": 1,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{ "op": "take",
"item": -1,
"item_name": "default"},
{ "op": "chooseleaf_firstn",
"num": 0,
"type": "host"},
{ "op": "emit"}]}10.4. Add an Erasure Code Rule
To add a CRUSH rule for use with an erasure coded pool, you may specify a rule name and an erasure code profile.
ceph osd crush rule create-erasure {rulename} {profilename}10.5. Remove a Rule
To remove a rule, execute the following and specify the CRUSH rule name:
ceph osd crush rule rm {name}