Chapter 7. Scaling hyperconverged nodes

If you are using director-deployed Red Hat Ceph Storage, it is important to understand the impact of replacing Controller nodes. The Ceph Monitor service runs on the Controller nodes and is typically assigned IP addresses from the Storage network. These Ceph Monitor service IP addresses are associated with VM instances where Red Hat Ceph Storage is used. They are not dynamically updated if the Ceph Monitor service IP address changes during replacement of the Controller node. This could result in a storage outage, especially if multiple Controller nodes are replaced. Each VM instance would have to be migrated, rebooted, or shelved and unshelved to resolve the IP address change and the resulting outage.

Reuse the IP addresses of the removed Ceph Monitor service instances instead of using new IP addresses to avoid this situation.

For an example, see the fixed_ip configuration example in Step 5 of Provisioning bare metal nodes for the overcloud.

Use the following command on a Controller node to find the current Ceph Monitor service IP addresses:

$ sudo cephadm shell -- ceph mon stat