Chapter 16. Managing a Red Hat Ceph Storage cluster using cephadm-ansible modules

Procedure

1. Log in to the Ansible administration node.

2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

   Example

   [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

3. Create the hosts file and add hosts, labels, and monitor IP address of the first host in the storage cluster:

   Syntax

   sudo vi INVENTORY_FILE
   
   HOST1 labels="['LABEL1', 'LABEL2']"
   HOST2 labels="['LABEL1', 'LABEL2']"
   HOST3 labels="['LABEL1']"
   
   [admin]
   ADMIN_HOST monitor_address=MONITOR_IP_ADDRESS labels="['ADMIN_LABEL', 'LABEL1', 'LABEL2']"

   Example

   [ceph-admin@admin cephadm-ansible]$ sudo vi hosts
   
   host02 labels="['mon', 'mgr']"
   host03 labels="['mon', 'mgr']"
   host04 labels="['osd']"
   host05 labels="['osd']"
   host06 labels="['osd']"
   
   [admin]
   host01 monitor_address=10.10.128.68 labels="['_admin', 'mon', 'mgr']"

4. Run the preflight playbook:

   Syntax

   ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=rhcs"

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=rhcs"

5. Create a playbook to bootstrap your cluster:

   Syntax

   sudo vi PLAYBOOK_FILENAME.yml
   
   ---
   - name: NAME_OF_PLAY
     hosts: BOOTSTRAP_HOST
     become: USE_ELEVATED_PRIVILEGES
     gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
     tasks:
       -name: NAME_OF_TASK
        cephadm_registry_login:
          state: STATE
          registry_url: REGISTRY_URL
          registry_username: REGISTRY_USER_NAME
          registry_password: REGISTRY_PASSWORD
   
       - name: NAME_OF_TASK
         cephadm_bootstrap:
           mon_ip: "{{ monitor_address }}"
           dashboard_user: DASHBOARD_USER
           dashboard_password: DASHBOARD_PASSWORD
           allow_fqdn_hostname: ALLOW_FQDN_HOSTNAME
           cluster_network: NETWORK_CIDR

   Example

   [ceph-admin@admin cephadm-ansible]$ sudo vi bootstrap.yml
   
   ---
   - name: bootstrap the cluster
     hosts: host01
     become: true
     gather_facts: false
     tasks:
       - name: login to registry
         cephadm_registry_login:
           state: login
           registry_url: registry.redhat.io
           registry_username: user1
           registry_password: mypassword1
   
       - name: bootstrap initial cluster
         cephadm_bootstrap:
           mon_ip: "{{ monitor_address }}"
           dashboard_user: mydashboarduser
           dashboard_password: mydashboardpassword
           allow_fqdn_hostname: true
           cluster_network: 10.10.128.0/28

6. Run the playbook:

   Syntax

   ansible-playbook -i INVENTORY_FILE PLAYBOOK_FILENAME.yml -vvv

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts bootstrap.yml -vvv

Procedure

1. Use the following procedure to add new hosts to the cluster:

   1. Log in to the Ansible administration node.

   2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

      Example

      [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

   3. Add the new hosts and labels to the Ansible inventory file.

      Syntax

      sudo vi INVENTORY_FILE
      
      NEW_HOST1 labels="['LABEL1', 'LABEL2']"
      NEW_HOST2 labels="['LABEL1', 'LABEL2']"
      NEW_HOST3 labels="['LABEL1']"
      
      [admin]
      ADMIN_HOST monitor_address=MONITOR_IP_ADDRESS labels="['ADMIN_LABEL', 'LABEL1', 'LABEL2']"

      Example

      [ceph-admin@admin cephadm-ansible]$ sudo vi hosts
      
      host02 labels="['mon', 'mgr']"
      host03 labels="['mon', 'mgr']"
      host04 labels="['osd']"
      host05 labels="['osd']"
      host06 labels="['osd']"
      
      [admin]
      host01 monitor_address= 10.10.128.68 labels="['_admin', 'mon', 'mgr']"

      Note

      If you have previously added the new hosts to the Ansible inventory file and ran the preflight playbook on the hosts, skip to step 3.

   4. Run the preflight playbook with the --limit option:

      Syntax

      ansible-playbook -i INVENTORY_FILE cephadm-preflight.yml --extra-vars "ceph_origin=rhcs" --limit NEWHOST

      Example

      [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts cephadm-preflight.yml --extra-vars "ceph_origin=rhcs" --limit host02

      The preflight playbook installs podman, lvm2, chronyd, and cephadm on the new host. After installation is complete, cephadm resides in the /usr/sbin/ directory.

   5. Create a playbook to add the new hosts to the cluster:

      Syntax

      sudo vi PLAYBOOK_FILENAME.yml
      
      ---
      - name: PLAY_NAME
        hosts: HOSTS_OR_HOST_GROUPS
        become: USE_ELEVATED_PRIVILEGES
        gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
        tasks:
          - name: NAME_OF_TASK
            ceph_orch_host:
              name: "{{ ansible_facts['hostname'] }}"
              address: "{{ ansible_facts['default_ipv4']['address'] }}"
              labels: "{{ labels }}"
            delegate_to: HOST_TO_DELEGATE_TASK_TO
      
          - name: NAME_OF_TASK
            when: inventory_hostname in groups['admin']
            ansible.builtin.shell:
              cmd: CEPH_COMMAND_TO_RUN
            register: REGISTER_NAME
      
          - name: NAME_OF_TASK
            when: inventory_hostname in groups['admin']
            debug:
              msg: "{{ REGISTER_NAME.stdout }}"

      Note

      By default, Ansible executes all tasks on the host that matches the hosts line of your playbook. The ceph orch commands must run on the host that contains the admin keyring and the Ceph configuration file. Use the delegate_to keyword to specify the admin host in your cluster.

      Example

      [ceph-admin@admin cephadm-ansible]$ sudo vi add-hosts.yml
      
      ---
      - name: add additional hosts to the cluster
        hosts: all
        become: true
        gather_facts: true
        tasks:
          - name: add hosts to the cluster
            ceph_orch_host:
              name: "{{ ansible_facts['hostname'] }}"
              address: "{{ ansible_facts['default_ipv4']['address'] }}"
              labels: "{{ labels }}"
            delegate_to: host01
      
          - name: list hosts in the cluster
            when: inventory_hostname in groups['admin']
            ansible.builtin.shell:
              cmd: ceph orch host ls
            register: host_list
      
          - name: print current list of hosts
            when: inventory_hostname in groups['admin']
            debug:
              msg: "{{ host_list.stdout }}"

      In this example, the playbook adds the new hosts to the cluster and displays a current list of hosts.

   6. Run the playbook to add additional hosts to the cluster:

      Syntax

      ansible-playbook -i INVENTORY_FILE PLAYBOOK_FILENAME.yml

      Example

      [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts add-hosts.yml

2. Use the following procedure to remove hosts from the cluster:

   1. Log in to the Ansible administration node.

   2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

      Example

      [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

   3. Create a playbook to remove a host or hosts from the cluster:

      Syntax

      sudo vi PLAYBOOK_FILENAME.yml
      
      ---
      - name: NAME_OF_PLAY
        hosts: ADMIN_HOST
        become: USE_ELEVATED_PRIVILEGES
        gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
        tasks:
          - name: NAME_OF_TASK
            ceph_orch_host:
              name: HOST_TO_REMOVE
              state: STATE
      
          - name: NAME_OF_TASK
            ceph_orch_host:
              name: HOST_TO_REMOVE
              state: STATE
            retries: NUMBER_OF_RETRIES
            delay: DELAY
            until: CONTINUE_UNTIL
            register: REGISTER_NAME
      
          - name: NAME_OF_TASK
            ansible.builtin.shell:
              cmd: ceph orch host ls
            register: REGISTER_NAME
      
          - name: NAME_OF_TASK
              debug:
                msg: "{{ REGISTER_NAME.stdout }}"

      Example

      [ceph-admin@admin cephadm-ansible]$ sudo vi remove-hosts.yml
      
      ---
      - name: remove host
        hosts: host01
        become: true
        gather_facts: true
        tasks:
          - name: drain host07
            ceph_orch_host:
              name: host07
              state: drain
      
          - name: remove host from the cluster
            ceph_orch_host:
              name: host07
              state: absent
            retries: 20
            delay: 1
            until: result is succeeded
            register: result
      
           - name: list hosts in the cluster
             ansible.builtin.shell:
               cmd: ceph orch host ls
             register: host_list
      
           - name: print current list of hosts
             debug:
               msg: "{{ host_list.stdout }}"

      In this example, the playbook tasks drain all daemons on host07, removes the host from the cluster, and displays a current list of hosts.

   4. Run the playbook to remove host from the cluster:

      Syntax

      ansible-playbook -i INVENTORY_FILE PLAYBOOK_FILENAME.yml

      Example

      [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts remove-hosts.yml

Procedure

1. Log in to the Ansible administration node.

2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

   Example

   [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

3. Create a playbook with configuration changes:

   Syntax

   sudo vi PLAYBOOK_FILENAME.yml
   
   ---
   - name: PLAY_NAME
     hosts: ADMIN_HOST
     become: USE_ELEVATED_PRIVILEGES
     gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
     tasks:
       - name: NAME_OF_TASK
         ceph_config:
           action: GET_OR_SET
           who: DAEMON_TO_SET_CONFIGURATION_TO
           option: CEPH_CONFIGURATION_OPTION
           value: VALUE_OF_PARAMETER_TO_SET
   
       - name: NAME_OF_TASK
         ceph_config:
           action: GET_OR_SET
           who: DAEMON_TO_SET_CONFIGURATION_TO
           option: CEPH_CONFIGURATION_OPTION
         register: REGISTER_NAME
   
       - name: NAME_OF_TASK
         debug:
           msg: "MESSAGE_TO_DISPLAY {{ REGISTER_NAME.stdout }}"

   Example

   [ceph-admin@admin cephadm-ansible]$ sudo vi change_configuration.yml
   
   ---
   - name: set pool delete
     hosts: host01
     become: true
     gather_facts: false
     tasks:
       - name: set the allow pool delete option
         ceph_config:
           action: set
           who: mon
           option: mon_allow_pool_delete
           value: true
   
       - name: get the allow pool delete setting
         ceph_config:
           action: get
           who: mon
           option: mon_allow_pool_delete
         register: verify_mon_allow_pool_delete
   
       - name: print current mon_allow_pool_delete setting
         debug:
           msg: "the value of 'mon_allow_pool_delete' is {{ verify_mon_allow_pool_delete.stdout }}"

   In this example, the playbook first sets the mon_allow_pool_delete option to false. The playbook then gets the current mon_allow_pool_delete setting and displays the value in the Ansible output.

4. Run the playbook:

   Syntax

   ansible-playbook -i INVENTORY_FILE _PLAYBOOK_FILENAME.yml

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts change_configuration.yml

Procedure

1. Log in to the Ansible administration node.

2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

   Example

   [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

3. Create a playbook with the service specifications:

   Syntax

   sudo vi PLAYBOOK_FILENAME.yml
   
   ---
   - name: PLAY_NAME
     hosts: HOSTS_OR_HOST_GROUPS
     become: USE_ELEVATED_PRIVILEGES
     gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
     tasks:
       - name: NAME_OF_TASK
         ceph_orch_apply:
           spec: |
             service_type: SERVICE_TYPE
             service_id: UNIQUE_NAME_OF_SERVICE
             placement:
               host_pattern: 'HOST_PATTERN_TO_SELECT_HOSTS'
               label: LABEL
             spec:
               SPECIFICATION_OPTIONS:

   Example

   [ceph-admin@admin cephadm-ansible]$ sudo vi deploy_osd_service.yml
   
   ---
   - name: deploy osd service
     hosts: host01
     become: true
     gather_facts: true
     tasks:
       - name: apply osd spec
         ceph_orch_apply:
           spec: |
             service_type: osd
             service_id: osd
             placement:
               host_pattern: '*'
               label: osd
             spec:
               data_devices:
                 all: true

   In this example, the playbook deploys the Ceph OSD service on all hosts with the label osd.

4. Run the playbook:

   Syntax

   ansible-playbook -i INVENTORY_FILE _PLAYBOOK_FILENAME.yml

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts deploy_osd_service.yml

Procedure

1. Log in to the Ansible administration node.

2. Navigate to the /usr/share/cephadm-ansible directory on the Ansible administration node:

   Example

   [ceph-admin@admin ~]$ cd /usr/share/cephadm-ansible

3. Create a playbook with daemon state changes:

   Syntax

   sudo vi PLAYBOOK_FILENAME.yml
   
   ---
   - name: PLAY_NAME
     hosts: ADMIN_HOST
     become: USE_ELEVATED_PRIVILEGES
     gather_facts: GATHER_FACTS_ABOUT_REMOTE_HOSTS
     tasks:
       - name: NAME_OF_TASK
         ceph_orch_daemon:
           state: STATE_OF_SERVICE
           daemon_id: DAEMON_ID
           daemon_type: TYPE_OF_SERVICE

   Example

   [ceph-admin@admin cephadm-ansible]$ sudo vi restart_services.yml
   
   ---
   - name: start and stop services
     hosts: host01
     become: true
     gather_facts: false
     tasks:
       - name: start osd.0
         ceph_orch_daemon:
           state: started
           daemon_id: 0
           daemon_type: osd
   
       - name: stop mon.host02
         ceph_orch_daemon:
           state: stopped
           daemon_id: host02
           daemon_type: mon

   In this example, the playbook starts the OSD with an ID of 0 and stops a Ceph Monitor with an id of host02.

4. Run the playbook:

   Syntax

   ansible-playbook -i INVENTORY_FILE _PLAYBOOK_FILENAME.yml

   Example

   [ceph-admin@admin cephadm-ansible]$ ansible-playbook -i hosts restart_services.yml