6장. Removing a DCN node set

Procedure

1. Delete the host aggregate for the DCN site that you want to remove, for example, “az2”:

   1. Access the remote shell for the OpenStackClient pod from your workstation:

      $ oc rsh -n openstack openstackclient

   2. Optional: View the Compute nodes assigned to the host aggregate::

      # openstack aggregate show <aggregate_name>

   3. To remove the assigned Compute nodes from the host aggregate, enter the following command for each Compute node:

      # openstack aggregate remove host <aggregate_name> \
      <host_name>

      • Replace <aggregate_name> with the aggregate that corresponds to the DCN site to be removed, for example "az2".
      • Replace <host_name> with each host in the aggregate being removed, in turn.

   4. Remove the aggregate:

      openstack aggregate delete <aggregate_name>

      • Replace <aggregate_name> with the aggregate that corresponds to the DCN site to be removed, for example "az2".

   5. Exit the openstackclient pod:

      $ exit

2. Optional: Remove the cell:

   1. Open your OpenStackControlPlane CR file, openstack_control_plane.yaml, on your workstation.

   2. Remove the cell definition from the cellTemplates:

            cellTemplates:
              cell0:
                hasAPIAccess: true
                cellDatabaseAccount: nova-cell0
                cellDatabaseInstance: openstack
                cellMessageBusInstance: rabbitmq
              cell1:
                hasAPIAccess: true
                cellDatabaseAccount: nova-cell1
                cellDatabaseInstance: openstack-cell1
                cellMessageBusInstance: rabbitmq-cell1
              cell2:
                hasAPIAccess: true
                cellDatabaseAccount: nova-cell2
                cellDatabaseInstance: openstack-cell2
                cellMessageBusInstance: rabbitmq-cell2
      -       cell3:
      -         hasAPIAccess: true
      -         cellDatabaseAccount: nova-cell3
      -         cellDatabaseInstance: openstack-cell3
      -         cellMessageBusInstance: rabbitmq-cell3

   3. Delete the cell-specific RabbitMQ definition from the OpenStackControlPlane CR:

      spec:
      ...
        rabbitmq:
          templates:
            ...
            rabbitmq-<cellname>:
              ...

   4. Delete the cell-specific Galera definition from the `OpenStackControlPlane CR file:

      spec:
      ...
        galera:
          templates:
            ...
            openstack-<cellname>:
              ...

   5. Update the control plane:

      $ oc apply -f openstack_control_plane.yaml -n openstack

3. Remove the Block storage (cinder) pods for the DCN site:

   1. Get a list of volumes:

      $ openstack volume service list

      Example

      +------------------+--------------------------+------+---------+-------+----------------------------+
      | Binary           | Host                     | Zone | Status  | State | Updated At                 |
      +------------------+--------------------------+------+---------+-------+----------------------------+
      | cinder-scheduler | cinder-e479e-scheduler-0 | nova | enabled | down  | 2024-11-10T16:29:40.000000 |
      | cinder-scheduler | cinder-scheduler-0       | nova | enabled | up    | 2024-11-12T19:11:08.000000 |
      | cinder-volume    | cinder-volume-az0-0@ceph | az0  | enabled | up    | 2024-11-12T19:11:09.000000 |
      | cinder-backup    | cinder-backup-0          | nova | enabled | up    | 2024-11-12T19:11:08.000000 |
      | cinder-backup    | cinder-backup-1          | nova | enabled | up    | 2024-11-12T19:11:13.000000 |
      | cinder-backup    | cinder-backup-2          | nova | enabled | up    | 2024-11-12T19:11:16.000000 |
      | cinder-volume    | cinder-volume-az1-0@ceph | az1  | enabled | up    | 2024-11-12T19:11:15.000000 |
      | cinder-volume    | cinder-volume-az2-0@ceph | az2  | enabled | up    | 2024-11-12T17:28:28.000000 |
      +------------------+--------------------------+------+---------+-------+----------------------------+

   2. Disable the Block storage volume service of the availability zone(AZ) being removed:

      $ openstack volume service set \
      --disable cinder-volume-az2-0@ceph cinder-volume

   3. Verify that the Block storage volume is disabled:

      openstack volume service list

      Example

      +------------------+--------------------------+------+----------+-------+----------------------------+
      | Binary           | Host                     | Zone | Status   | State | Updated At                 |
      +------------------+--------------------------+------+----------+-------+----------------------------+
      | cinder-scheduler | cinder-e479e-scheduler-0 | nova | enabled  | down  | 2024-11-10T16:29:40.000000 |
      | cinder-scheduler | cinder-scheduler-0       | nova | enabled  | up    | 2024-11-12T19:23:38.000000 |
      | cinder-volume    | cinder-volume-az0-0@ceph | az0  | enabled  | up    | 2024-11-12T19:23:29.000000 |
      | cinder-backup    | cinder-backup-0          | nova | enabled  | up    | 2024-11-12T19:23:38.000000 |
      | cinder-backup    | cinder-backup-1          | nova | enabled  | up    | 2024-11-12T19:23:33.000000 |
      | cinder-backup    | cinder-backup-2          | nova | enabled  | up    | 2024-11-12T19:23:36.000000 |
      | cinder-volume    | cinder-volume-az1-0@ceph | az1  | enabled  | up    | 2024-11-12T19:23:35.000000 |
      | cinder-volume    | cinder-volume-az2-0@ceph | az2  | disabled | up    | 2024-11-12T19:23:24.000000 |
      +------------------+--------------------------+------+----------+-------+----------------------------+

   4. Open the OpenStackControlPlane manifest file, openstack_control_plane.yaml. Remove the CinderVolume pod for the site being removed, as shown in the following example:

           cinderVolumes:
              az2:
                customServiceConfig: |
                  [DEFAULT]
                  enabled_backends = ceph
                  glance_api_servers = https://glance-az2-internal.openstack.svc:9292
                  [ceph]
                  volume_backend_name = ceph
                  volume_driver = cinder.volume.drivers.rbd.RBDDriver
                  rbd_ceph_conf = /etc/ceph/az2.conf
                  rbd_user = openstack
                  rbd_pool = volumes
                  rbd_flatten_volume_from_snapshot = False
                  rbd_secret_uuid = 795dcbca-e715-5ac3-9b7e-a3f5c64eb89f
                  rbd_cluster_name = az2
                  backend_availability_zone = az2

4. Remove the cinder volume service for the DCN site you are removing:

   1. Open a shell to the cinder scheduler pod:

      oc rsh cinder-scheduler-0

   2. Remove the cinder volume service:

      cinder-manage service remove cinder-volume cinder-volume-az2-0@ceph

   3. Exit the shell

      $ exit

5. Remove the GlanceAPI pod for the site being removed:

   1. In the openstack-control-plane.yaml custom resource (CR) file, remove the az2 field and all fields under it:

           glanceAPIs:
              az0:
                <...>
              az1:
                <...>
             az2:
                apiTimeout: 60
                customServiceConfig: |
                  [DEFAULT]
                  enabled_import_methods = [web-download,copy-image,glance-direct]
                  enabled_backends = az0:rbd,az2:rbd
                  [glance_store]
                  default_backend = az2
                  [az0]
                  rbd_store_ceph_conf = /etc/ceph/az0.conf
                  store_description = "az0 RBD backend"
                  rbd_store_pool = images
                  rbd_store_user = openstack
                  rbd_thin_provisioning = True
                  [az2]
                  rbd_store_ceph_conf = /etc/ceph/az2.conf
                  store_description = "az2 RBD backend"
                  rbd_store_pool = images
                  rbd_store_user = openstack
                  rbd_thin_provisioning = True
                imageCache:
                  cleanerScheduler: '*/30 * * * *'
                  prunerScheduler: 1 0 * * *
                  size: ""
                networkAttachments:
                - storage
                override:
                  service:
                    internal:
                      metadata:
                        annotations:
                          metallb.universe.tf/address-pool: internalapi
                          metallb.universe.tf/allow-shared-ip: internalapi
                          metallb.universe.tf/loadBalancerIPs: 172.17.0.82
                      spec:
                        type: LoadBalancer
                replicas: 1
                resources: {}
                storage: {}
                tls:
                  api:
                    internal: {}
                    public: {}
                type: edge

   2. Reapply the control plane:

      oc apply -f openstack-control-plane.yaml

   3. Ensure that the Image service (glance) pods have been removed:

      $ oc get pods | grep glance | grep -v purge

      Example

      glance-e479e-az0-external-api-0           3/3     Running     0             2d
      glance-e479e-az0-external-api-1           3/3     Running     0             2d
      glance-e479e-az0-external-api-2           3/3     Running     0             2d
      glance-e479e-az0-internal-api-0           3/3     Running     0             2d
      glance-e479e-az0-internal-api-1           3/3     Running     0             2d
      glance-e479e-az0-internal-api-2           3/3     Running     0             2d
      glance-e479e-az1-edge-api-0

      참고

      glance-e4793-az2-edge-api-0 does not appear in this list.

   4. Ensure that the az2 pods are removed:

      $ oc get pods | grep cinder-volume

      Example

      cinder-volume-az0-0       2/2     Running     0   2d
      cinder-volume-az1-0       2/2     Running     0   2d

6. Remove the Ceph cluster from the DCN site:

   1. Shut down the Ceph clusters, but do not power off the hosts. For more information, see "Powering down and rebooting the cluster using the Ceph Orchestrator" in the Red Hat Ceph Storage Administration Guide:

      • Red Hat Ceph Storage 7 Administration Guide
      • Red Hat Ceph Storage 8 Administration Guide

      • Red Hat Ceph Storage 9 Administration Guide

        참고

        The hosts must remain powered on to complete the following steps.

   2. Remove the secret that was used for accessing the removed Ceph cluster:

      $ oc delete secret ceph-conf-az-2 -n openstack

   3. Re-create the secret for the central site so that it does not contain the secret for the Red Hat Ceph Storage cluster at az2. For example, if you have a three availability zones, az0, az1, and az2, and you are removing the edge location that corresponds to az2, run the following:

      oc delete secret ceph-conf-az-0 -n openstack
      oc create secret generic ceph-conf-az-0 \
      --from-file=az0.client.openstack.keyring \
      --from-file=az0.conf \
      --from-file=az1.client.openstack.keyring \
      --from-file=az1.conf -n openstack

   4. Edit the extraMounts in the OpenStackControlPlane to remove the reference to the AZ being removed and the removed secret. For example, for AZ2, remove the following list element:

              - propagation:
                - az2
                extraVolType: Ceph
                volumes:
                - name: ceph
                  secret:
                    name: ceph-conf-az-2

7. Remove the node set. To remove the node set that corresponds to the az2 availability zone, complete the steps in Removing an OpenStackDataPlaneNodeSet resource.