7.2. 将监控堆栈组件迁移到现有 Red Hat Ceph Storage 集群中的新节点

流程

1. 将监控标签添加到集群中的所有 Red Hat Ceph Storage 或 ComputeHCI 节点：

   for item in $(sudo cephadm shell --  ceph orch host ls --format json | jq -r '.[].hostname'); do
       sudo cephadm shell -- ceph orch host label add  $item monitoring;
   done

   Copy to Clipboard Toggle word wrap

2. 验证目标节点上的所有主机是否具有监控标签：

   [tripleo-admin@controller-0 ~]$ sudo cephadm shell -- ceph orch host ls
   
   HOST                        ADDR           LABELS
   cephstorage-0.redhat.local  192.168.24.11  osd monitoring
   cephstorage-1.redhat.local  192.168.24.12  osd monitoring
   cephstorage-2.redhat.local  192.168.24.47  osd monitoring
   controller-0.redhat.local   192.168.24.35  _admin mon mgr monitoring
   controller-1.redhat.local   192.168.24.53  mon _admin mgr monitoring
   controller-2.redhat.local   192.168.24.10  mon _admin mgr monitoring

   Copy to Clipboard Toggle word wrap

3. 从 Controller 节点中删除标签：

   $ for i in 0 1 2; do sudo cephadm shell -- ceph orch host label rm "controller-$i.redhat.local" monitoring; done
   
   Removed label monitoring from host controller-0.redhat.local
   Removed label monitoring from host controller-1.redhat.local
   Removed label monitoring from host controller-2.redhat.local

   Copy to Clipboard Toggle word wrap

4. 转储当前的监控堆栈规格：

   function export_spec {
       local component="$1"
       local target_dir="$2"
       sudo cephadm shell -- ceph orch ls --export "$component" > "$target_dir/$component"
   }
   
   SPEC_DIR=${SPEC_DIR:-"$PWD/ceph_specs"}
   mkdir -p ${SPEC_DIR}
   for m in grafana prometheus alertmanager; do
       export_spec "$m" "$SPEC_DIR"
   done

   Copy to Clipboard Toggle word wrap

5. 对于每个守护进程，编辑当前的 spec，并将 placement.hosts: 部分替换为 placement.label: 部分，例如：

   service_type: grafana
   service_name: grafana
   placement:
     label: monitoring
   networks:
   - 172.17.3.0/24
   spec:
     port: 3100

   Copy to Clipboard Toggle word wrap

   此步骤也适用于 Prometheus 和 Alertmanager 规格。

6. 应用新的监控 spec 来重新定位监控堆栈守护进程：

   SPEC_DIR=${SPEC_DIR:-"$PWD/ceph_specs"}
   function migrate_daemon {
       local component="$1"
       local target_dir="$2"
       sudo cephadm shell -m "$target_dir" -- ceph orch apply -i /mnt/ceph_specs/$component
   }
   for m in grafana prometheus alertmanager; do
       migrate_daemon  "$m" "$SPEC_DIR"
   done

   Copy to Clipboard Toggle word wrap

7. 验证守护进程是否已部署到预期的节点上：

   [ceph: root@controller-0 /]# ceph orch ps | grep -iE "(prome|alert|grafa)"
   alertmanager.cephstorage-2  cephstorage-2.redhat.local  172.17.3.144:9093,9094
   grafana.cephstorage-0       cephstorage-0.redhat.local  172.17.3.83:3100
   prometheus.cephstorage-1    cephstorage-1.redhat.local  172.17.3.53:9092

   Copy to Clipboard Toggle word wrap
   注意

   迁移监控堆栈后，您会丢失高可用性。监控堆栈守护进程不再具有虚拟 IP 地址和 HAProxy。节点导出器仍然在所有节点上运行。

8. 检查 Red Hat Ceph Storage 配置，以确保它与目标节点上的配置一致。特别是，请关注以下配置条目：

   [ceph: root@controller-0 /]# ceph config dump | grep -i dashboard
   ...
   mgr  advanced  mgr/dashboard/ALERTMANAGER_API_HOST  http://172.17.3.83:9093
   mgr  advanced  mgr/dashboard/GRAFANA_API_URL        https://172.17.3.144:3100
   mgr  advanced  mgr/dashboard/PROMETHEUS_API_HOST    http://172.17.3.83:9092
   mgr  advanced  mgr/dashboard/controller-0.ycokob/server_addr  172.17.3.33
   mgr  advanced  mgr/dashboard/controller-1.lmzpuc/server_addr  172.17.3.147
   mgr  advanced  mgr/dashboard/controller-2.xpdgfl/server_addr  172.17.3.138

   Copy to Clipboard Toggle word wrap

9. 验证 grafana、alertmanager 和 prometheus 服务的 API_HOST/URL 指向每个守护进程重新定位节点的存储网络上的 IP 地址：

   [ceph: root@controller-0 /]# ceph orch ps | grep -iE "(prome|alert|grafa)"
   alertmanager.cephstorage-0  cephstorage-0.redhat.local  172.17.3.83:9093,9094
   alertmanager.cephstorage-1  cephstorage-1.redhat.local  172.17.3.53:9093,9094
   alertmanager.cephstorage-2  cephstorage-2.redhat.local  172.17.3.144:9093,9094
   grafana.cephstorage-0       cephstorage-0.redhat.local  172.17.3.83:3100
   grafana.cephstorage-1       cephstorage-1.redhat.local  172.17.3.53:3100
   grafana.cephstorage-2       cephstorage-2.redhat.local  172.17.3.144:3100
   prometheus.cephstorage-0    cephstorage-0.redhat.local  172.17.3.83:9092
   prometheus.cephstorage-1    cephstorage-1.redhat.local  172.17.3.53:9092
   prometheus.cephstorage-2    cephstorage-2.redhat.local  172.17.3.144:9092

   Copy to Clipboard Toggle word wrap

   [ceph: root@controller-0 /]# ceph config dump
   ...
   ...
   mgr  advanced  mgr/dashboard/ALERTMANAGER_API_HOST   http://172.17.3.83:9093
   mgr  advanced  mgr/dashboard/PROMETHEUS_API_HOST     http://172.17.3.83:9092
   mgr  advanced  mgr/dashboard/GRAFANA_API_URL         https://172.17.3.144:3100

   Copy to Clipboard Toggle word wrap
   注意

   Ceph 控制面板，作为 Ceph mgr 提供的服务，不受重新定位的影响。当迁移活跃 mgr 守护进程或强制失败时，您可能会遇到影响。但是，您可以在 Ceph Manager 配置中定义 3 个副本，将请求重定向到不同的实例。