Chapter 4. Pools

To manage pools, you can list, create, and remove pools. You can also view the utilization statistics for each pool.

To list your cluster’s pools, run:

[ceph: root@host01 /]# ceph osd lspools

Before creating pools, see the Pool, PG and CRUSH Configuration Reference chapter in the Configuration Guide for Red Hat Ceph Storage 5.

It is better to adjust the default value for the number of placement groups in the Ceph configuration file, because the default value does not have to suit your needs.

# osd pool default pg num = 100
# osd pool default pgp num = 100

To create a replicated pool, run:

ceph osd pool create POOL_NAME PG_NUMBER PGP_NUMBER [replicated] \ [crush-rule-name] [expected-num-objects]

To create a bulk pool, run:

ceph osd pool create POOL_NAME --bulk

To create an erasure-coded pool, run:

ceph osd pool create POOL_NAME PG_NUMBER PGP_NUMBER erasure \ [erasure-code-profile] [crush-rule-name] [expected-num-objects]

Where:

When you create a pool, set the number of placement groups to a reasonable value, for example to 100. Consider the total number of placement groups per OSD. Placement groups are computationally expensive, so performance will degrade when you have many pools with many placement groups, for example, 50 pools with 100 placement groups each. The point of diminishing returns depends upon the power of the OSD host.

You can set pool quotas for the maximum number of bytes and the maximum number of objects per pool.

ceph osd pool set-quota POOL_NAME [max_objects OBJECT_COUNT] [max_bytes BYTES]

[ceph: root@host01 /]# ceph osd pool set-quota data max_objects 10000

To remove a quota, set its value to 0.

To delete a pool, run:

ceph osd pool delete POOL_NAME [POOL_NAME --yes-i-really-really-mean-it]

If a pool has its own rule, consider removing it after deleting the pool. If a pool has users strictly for its own use, consider deleting those users after deleting the pool.

To rename a pool, run:

ceph osd pool rename CURRENT_POOL_NAME NEW_POOL_NAME

If you rename a pool and you have per-pool capabilities for an authenticated user, you must update the user’s capabilities, that is caps, with the new pool name.

Sometimes it is necessary to migrate all objects from one pool to another. This is done in cases such as needing to change parameters that cannot be modified on a specific pool. For example, needing to reduce the number of placement groups of a pool.

The migration methods described for Ceph Block Device are more recommended than those documented here. using the cppool does not preserve all snapshots and snapshot related metadata, resulting in an unfaithful copy of the data. For example, copying an RBD pool does not completely copy the image. In this case, snaps are not present and will not work properly. The cppool also does not preserve the user_version field that some librados users may rely on.

If migrating a pool is necessary and your user workloads contain images other than Ceph Block Devices, continue with one of the procedures documented here.

Copy all objects with the rados cppool command.

ceph osd pool create NEW_POOL PG_NUM [ <other new pool parameters> ]
rados cppool SOURCE_POOL NEW_POOL
ceph osd pool rename SOURCE_POOL NEW_SOURCE_POOL_NAME
ceph osd pool rename NEW_POOL SOURCE_POOL

[ceph: root@host01 /]# ceph osd pool create pool1 250
[ceph: root@host01 /]# rados cppool pool2 pool1
[ceph: root@host01 /]# ceph osd pool rename pool2 pool3
[ceph: root@host01 /]# ceph osd pool rename pool1 pool2

Method two - using a local drive

To show a pool’s utilization statistics, run:

[ceph: root@host01 /] rados df

To set a value to a pool, run the following command:

ceph osd pool set POOL_NAME KEY VALUE

The Pool Values section lists all key-values pairs that you can set.

To get a value from a pool, run the following command:

ceph osd pool get POOL_NAME KEY

The Pool Values section lists all key-values pairs that you can get.

Red Hat Ceph Storage provides additional protection for pools to prevent unauthorized types of clients from writing data to the pool. This means that system administrators must expressly enable pools to receive I/O operations from Ceph Block Device, Ceph Object Gateway, Ceph Filesystem, or for a custom application.

To enable a client application to conduct I/O operations on a pool, run the following:

ceph osd pool application enable POOL_NAME APPLICATION {--yes-i-really-mean-it}

Where APPLICATION is:

{
    "checks": {
        "POOL_APP_NOT_ENABLED": {
            "severity": "HEALTH_WARN",
            "summary": {
                "message": "application not enabled on 1 pool(s)"
            },
            "detail": [
                {
                    "message": "application not enabled on pool '<pool-name>'"
                },
                {
                    "message": "use 'ceph osd pool application enable <pool-name> <app-name>', where <app-name> is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."
                }
            ]
        }
    },
    "status": "HEALTH_WARN",
    "overall_status": "HEALTH_WARN",
    "detail": [
        "'ceph health' JSON format has changed in luminous. If you see this your monitoring system is scraping the wrong fields. Disable this with 'mon health preluminous compat warning = false'"
    ]
}

To disable a client application from conducting I/O operations on a pool, run the following:

ceph osd pool application disable POOL_NAME APPLICATION {--yes-i-really-mean-it}

Where APPLICATION is:

Provides the functionality to set key-value pairs describing attributes of the client application.

To set client application metadata on a pool, run the following:

ceph osd pool application set POOL_NAME APPLICATION KEY VALUE

Where APPLICATION is:

To remove client application metadata on a pool, run the following:

ceph osd pool application set POOL_NAME APPLICATION KEY

Where APPLICATION is:

To set the number of object replicas on a replicated pool, run the following command:

ceph osd pool set POOL_NAME size NUMBER_OF_REPLICAS

You can run this command for each pool.

[ceph: root@host01 /]# ceph osd pool set data size 3

[ceph: root@host01 /]# ceph osd pool set data min_size 2

To get the number of object replicas, run the following command:

[ceph: root@host01 /]# ceph osd dump | grep 'replicated size'

Ceph will list the pools with the replicated size attribute highlighted. By default, Ceph creates two replicas of an object, that is a total of three copies, or a size of 3.

The following list contains key-values pairs that you can set or get. For further information, see the Set Pool Values and Get Pool Values sections.