Chapter 3. Management of hosts using the Ceph Orchestrator

Procedure

1. From the Ceph administration node, log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. Extract the cluster’s public SSH keys to a folder:

   Syntax

   ceph cephadm get-pub-key > ~/PATH

   Example

   [ceph: root@host01 /]# ceph cephadm get-pub-key > ~/ceph.pub

3. Copy Ceph cluster’s public SSH keys to the root user’s authorized_keys file on the new host:

   Syntax

   ssh-copy-id -f -i ~/PATH root@HOST_NAME_2

   Example

   [ceph: root@host01 /]# ssh-copy-id -f -i ~/ceph.pub root@host02

4. From the Ansible administration node, add the new host to the Ansible inventory file. The default location for the file is /usr/share/cephadm-ansible/hosts. The following example shows the structure of a typical inventory file:

   Example

   host01
   host02
   host03
   
   [admin]
   host00

   Note

   If you have previously added the new host to the Ansible inventory file and run the preflight playbook on the host, skip to step 6.

5. Run the preflight playbook with the --limit option:

   Syntax

   ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=rhcs" --limit NEWHOST

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=rhcs" --limit host02

   The preflight playbook installs podman, lvm2, chronyd, and cephadm on the new host. After installation is complete, cephadm resides in the /usr/sbin/ directory.

6. From the Ceph administration node, log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

7. Use the cephadm orchestrator to add hosts to the storage cluster:

   Syntax

   ceph orch host add HOST_NAME IP_ADDRESS_OF_HOST [--label=LABEL_NAME_1,LABEL_NAME_2]

   The --label option is optional and this adds the labels when adding the hosts. You can add multiple labels to the host.

   Example

   [ceph: root@host01 /]# ceph orch host add host02 10.10.128.70 --labels=mon,mgr

Procedure

1. Create the hosts.yaml file:

   Example

   [root@host01 ~]# touch hosts.yaml

2. Edit the hosts.yaml file to include the following details:

   Example

   service_type: host
   addr: host01
   hostname: host01
   labels:
   - mon
   - osd
   - mgr
   ---
   service_type: host
   addr: host02
   hostname: host02
   labels:
   - mon
   - osd
   - mgr
   ---
   service_type: host
   addr: host03
   hostname: host03
   labels:
   - mon
   - osd

3. Mount the YAML file under a directory in the container:

   Example

   [root@host01 ~]# cephadm shell --mount hosts.yaml:/var/lib/ceph/hosts.yaml

4. Navigate to the directory:

   Example

   [ceph: root@host01 /]# cd /var/lib/ceph/

5. Deploy the hosts using service specification:

   Syntax

   ceph orch apply -i FILE_NAME.yaml

   Example

   [ceph: root@host01 hosts]# ceph orch apply -i hosts.yaml

Procedure

1. Log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. List the hosts of the cluster:

   Example

   [ceph: root@host01 /]# ceph orch host ls

   You will see that the STATUS of the hosts is blank which is expected.

Procedure

1. Log in to the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. Add a label to a host:

   Syntax

   ceph orch host label add HOSTNAME LABEL

   Example

   [ceph: root@host01 /]# ceph orch host label add host02 mon

Procedure

1. Launch the cephadm shell:

   [root@host01 ~]# cephadm shell
   [ceph: root@host01 /]#

2. Remove the label.

   Syntax

   ceph orch host label rm HOSTNAME LABEL

   Example

   [ceph: root@host01 /]# ceph orch host label rm host02 mon

Procedure

1. Log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. Fetch the host details:

   Example

   [ceph: root@host01 /]# ceph orch host ls

3. Drain all the daemons from the host:

   Syntax

   ceph orch host drain HOSTNAME

   Example

   [ceph: root@host01 /]# ceph orch host drain host02

   The _no_schedule label is automatically applied to the host which blocks deployment.

4. Check the status of OSD removal:

   Example

   [ceph: root@host01 /]# ceph orch osd rm status

   When no placement groups (PG) are left on the OSD, the OSD is decommissioned and removed from the storage cluster.

5. Check if all the daemons are removed from the storage cluster:

   Syntax

   ceph orch ps HOSTNAME

   Example

   [ceph: root@host01 /]# ceph orch ps host02

6. Remove the host:

   Syntax

   ceph orch host rm HOSTNAME

   Example

   [ceph: root@host01 /]# ceph orch host rm host02

1. Confirms the removal of hosts does not impact data availability by running the orch host ok-to-stop command.
2. If the host has Ceph OSD daemons, it applies noout to the host subtree to prevent data migration from triggering during the planned maintenance slot.
3. Stops the Ceph target, thereby, stopping all the daemons.
4. Disables the ceph target on the host, to prevent a reboot from automatically starting Ceph services.

Procedure

1. Log into the Cephadm shell:

   Example

   [root@host01 ~]# cephadm shell

2. You can either place the host in maintenance mode or place it out of the maintenance mode:

   • Place the host in maintenance mode:

     Syntax

     ceph orch host maintenance enter HOST_NAME [--force]

     Example

     [ceph: root@host01 /]# ceph orch host maintenance enter host02 --force

     The --force flag allows the user to bypass warnings, but not alerts.

   • Place the host out of the maintenance mode:

     Syntax

     ceph orch host maintenance exit HOST_NAME

     Example

     [ceph: root@host01 /]# ceph orch host maintenance exit host02