Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

此内容没有您所选择的语言版本。

Chapter 13. Add an OSD

When you have a cluster up and running, you may add OSDs or remove OSDs from the cluster at runtime.

A Ceph OSD generally consists of one ceph-osd daemon for one storage drive (and its associated journal) within a host machine. If your host has multiple storage drives, you may map one ceph-osd daemon for each drive.

Generally, it’s a good idea to check the capacity of your cluster to see if you are reaching the upper end of its capacity. As your cluster reaches its near full ratio, you should add one or more OSDs to expand your cluster’s capacity.

Important

Do not let your cluster reach its full ratio before adding an OSD. OSD failures that occur after the cluster reaches its near full ratio may cause the cluster to exceed its full ratio. Ceph will block write access to protect your data until you resolve storage capacity issues. DO NOT REMOVE OSDs without considering the impact on the full ratio first.

OSDs and their supporting hardware should be similarly configured as a storage strategy for the pool(s) that will use the OSDs. Ceph prefers uniform hardware across pools for a consistent performance profile. For best performance, consider a CRUSH hierarchy with drives of the same type/size. View the Storage Strategies guide for details.

If you are adding drives of dissimilar size, you can adjust their weights. When you add the OSD to the CRUSH map, consider the weight you give to the new OSD. Hard drive capacity grows approximately 40% per year, so newer OSD hosts may have larger hard drives than older hosts in the cluster (i.e., they may have greater weight).

13.1. ceph-deploy
复制链接

Before creating OSDs, consider the following:

  • We recommend using the XFS filesystem (default).
  • We recommend using SSDs for journals. It is common to partition SSDs to serve multiple OSDs. Ensure that the number of SSD partitions does not exceed your SSD’s sequential write limits. Also, ensure that SSD partitions are properly aligned, or their write performance will suffer.
  • We recommend using ceph-deploy disk zap on a Ceph OSD drive before executing ceph-deploy osd create. For example: 
ceph-deploy disk zap <ceph-node>:<data-drive>
Copy to Clipboard Toggle word wrap

From your admin node, use ceph-deploy to prepare the OSDs. 

ceph-deploy osd prepare <ceph-node>:<data-drive> <journal-partition> [<ceph-node>:<data-drive> <journal-partition>]
Copy to Clipboard Toggle word wrap

For example: 

ceph-deploy osd prepare node2:sdb ssdb node3:sdd ssdb node4:sdd ssdb
Copy to Clipboard Toggle word wrap

In the foregoing example, sdb is a spinning hard drive. Ceph will use the entire drive for OSD data. ssdb is a partition on an SSD drive, which Ceph will use to store the journal for the OSD.

Once you prepare OSDs, use ceph-deploy to activate the OSDs. 

ceph-deploy osd activate <ceph-node>:<data-drive> <journal-partition> [<ceph-node>:<data-drive> <journal-partition>]
Copy to Clipboard Toggle word wrap

For example: 

ceph-deploy osd activate node2:sdb ssdb node3:sdd ssdb node4:sdd ssdb
Copy to Clipboard Toggle word wrap

To achieve an active + clean state, you must add as many OSDs as the value of osd pool default size = <n> from your Ceph configuration file.

13.2. manual
复制链接

This procedure sets up a ceph-osd daemon, configures it to use one drive, and configures the cluster to distribute data to the OSD. If your host has multiple drives, you may add an OSD for each drive by repeating this procedure.

To add an OSD, create a data directory for it, mount a drive to that directory, add the OSD to the cluster, and then add it to the CRUSH map.

  1. Create the OSD. If no UUID is given, it will be set automatically when the OSD starts up. The following command will output the OSD number, which you will need for subsequent steps. : 

    ceph osd create [{uuid}]
    Copy to Clipboard Toggle word wrap

  2. Create the default directory on your new OSD. : 

    ssh {new-osd-host}
sudo mkdir /var/lib/ceph/osd/ceph-{osd-number}
    Copy to Clipboard Toggle word wrap

  3. If the OSD is for a drive other than the OS drive, prepare it for use with Ceph, and mount it to the directory you just created: 

    ssh {new-osd-host}
sudo mkfs -t {fstype} /dev/{drive}
sudo mount -o user_xattr /dev/{hdd} /var/lib/ceph/osd/ceph-{osd-number}
    Copy to Clipboard Toggle word wrap

  4. Initialize the OSD data directory. : 

    ssh {new-osd-host}
ceph-osd -i {osd-num} --mkfs --mkkey
    Copy to Clipboard Toggle word wrap

    The directory must be empty before you can run ceph-osd.

  5. Register the OSD authentication key. The value of ceph for ceph-{osd-num} in the path is the $cluster-$id. If your cluster name differs from ceph, use your cluster name instead.: 

    ceph auth add osd.{osd-num} osd 'allow *' mon 'allow rwx' -i /var/lib/ceph/osd/ceph-{osd-num}/keyring
    Copy to Clipboard Toggle word wrap

  6. Add the OSD to the CRUSH map so that the OSD can begin receiving data. The ceph osd crush add command allows you to add OSDs to the CRUSH hierarchy wherever you wish. If you specify at least one bucket, the command will place the OSD into the most specific bucket you specify, and it will move that bucket underneath any other buckets you specify. Important: If you specify only the root bucket, the command will attach the OSD directly to the root, but CRUSH rules expect OSDs to be inside of hosts.

    Execute the following: 

    ceph osd crush add {id-or-name} {weight}  [{bucket-type}={bucket-name} ...]
    Copy to Clipboard Toggle word wrap

    You may also decompile the CRUSH map, add the OSD to the device list, add the host as a bucket (if it’s not already in the CRUSH map), add the device as an item in the host, assign it a weight, recompile it and set it.

13.2.1. Starting the OSD
复制链接

After you add an OSD to Ceph, the OSD is in your configuration. However, it is not yet running. The OSD is down and in. You must start your new OSD before it can begin receiving data. For sysvinit, execute: 

sudo /etc/init.d/ceph start osd.{osd-num}
Copy to Clipboard Toggle word wrap

Once you start your OSD, it is up and in.

13.2.2. Observe the Data Migration
复制链接

Once you have added your new OSD to the CRUSH map, Ceph will begin rebalancing the server by migrating placement groups to your new OSD. You can observe this process with the ceph tool. : 

ceph -w
Copy to Clipboard Toggle word wrap

You should see the placement group states change from active+clean to active, some degraded objects, and finally active+clean when migration completes. (Control-c to exit.)

返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat