Chapter 1. Red Hat OpenStack Services on OpenShift 18.0 adoption overview

Procedure

1. Find the network configuration in the network_data.yaml file. For example:

   - name: InternalApi
     mtu: 1500
     vip: true
     vlan: 20
     name_lower: internal_api
     dns_domain: internal.mydomain.tld.
     service_net_map_replace: internal
     subnets:
       internal_api_subnet:
         ip_subnet: '172.17.0.0/24'
         allocation_pools: [{'start': '172.17.0.4', 'end': '172.17.0.250'}]

2. Retrieve the VLAN tag that is used in the vlan key and the IP range in the ip_subnet key for each isolated network from the network_data.yaml file. When reusing subnet ranges from the existing deployment for the new control plane services, the ranges are split into separate pools for control plane services and load-balancer IP addresses.

3. Use the tripleo-ansible-inventory.yaml file to determine the list of IP addresses that are already consumed in the adopted environment. For each listed host in the file, make a note of the IP and VIP addresses that are consumed by the node. For example:

   Standalone:
     hosts:
       standalone:
         ...
         internal_api_ip: 172.17.0.100
       ...
     ...
   standalone:
     children:
       Standalone: {}
     vars:
       ...
       internal_api_vip: 172.17.0.2
       ...

   Note

   In this example, the 172.17.0.2 and 172.17.0.100 values are consumed and are not available for the new control plane services until the adoption is complete.

4. Repeat this procedure for each isolated network and each host in the configuration.

Procedure

1. Gather the current status of the monitoring stack. Verify that the hosts have no monitoring label, or grafana, prometheus, or alertmanager, in cases of a per daemons placement evaluation:

   Note

   The entire relocation process is driven by cephadm and relies on labels to be assigned to the target nodes, where the daemons are scheduled. For more information about assigning labels to nodes, review the Red Hat Knowledgebase article Red Hat Ceph Storage: Supported configurations.

   [tripleo-admin@controller-0 ~]$ sudo cephadm shell -- ceph orch host ls
   
   HOST                    	ADDR       	LABELS                 	STATUS
   cephstorage-0.redhat.local  192.168.24.11  osd mds
   cephstorage-1.redhat.local  192.168.24.12  osd mds
   cephstorage-2.redhat.local  192.168.24.47  osd mds
   controller-0.redhat.local   192.168.24.35  _admin mon mgr
   controller-1.redhat.local   192.168.24.53  mon _admin mgr
   controller-2.redhat.local   192.168.24.10  mon _admin mgr
   6 hosts in cluster

   Confirm that the cluster is healthy and that both ceph orch ls and ceph orch ps return the expected number of deployed daemons.

2. Review and update the container image registry:

   Note

   If you run the Red Hat Ceph Storage externalization procedure after you migrate the Red Hat OpenStack Platform control plane, update the container images in the Red Hat Ceph Storage cluster configuration. The current container images point to the undercloud registry, which might not be available anymore. Because the undercloud is not available after adoption is complete, replace the undercloud-provided images with an alternative registry.

   $ ceph config dump
   ...
   ...
   mgr   advanced  mgr/cephadm/container_image_alertmanager    undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus-alertmanager:v4.10
   mgr   advanced  mgr/cephadm/container_image_base            undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhceph
   mgr   advanced  mgr/cephadm/container_image_grafana         undercloud-0.ctlplane.redhat.local:8787/rh-osbs/grafana:latest
   mgr   advanced  mgr/cephadm/container_image_node_exporter   undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus-node-exporter:v4.10
   mgr   advanced  mgr/cephadm/container_image_prometheus      undercloud-0.ctlplane.redhat.local:8787/rh-osbs/openshift-ose-prometheus:v4.10

3. Remove the undercloud container images:

   $ cephadm shell -- ceph config rm mgr mgr/cephadm/container_image_base \
   for i in prometheus grafana alertmanager node_exporter; do \
       cephadm shell -- ceph config rm mgr mgr/cephadm/container_image_$i \
   done

Procedure

1. Check the current status of the Red Hat Ceph Storage nodes:

   (undercloud) [stack@undercloud-0 ~]$ metalsmith list
   
   
       +------------------------+    +----------------+
       | IP Addresses           |    |  Hostname      |
       +------------------------+    +----------------+
       | ctlplane=192.168.24.25 |    | cephstorage-0  |
       | ctlplane=192.168.24.10 |    | cephstorage-1  |
       | ctlplane=192.168.24.32 |    | cephstorage-2  |
       | ctlplane=192.168.24.28 |    | compute-0      |
       | ctlplane=192.168.24.26 |    | compute-1      |
       | ctlplane=192.168.24.43 |    | controller-0   |
       | ctlplane=192.168.24.7  |    | controller-1   |
       | ctlplane=192.168.24.41 |    | controller-2   |
       +------------------------+    +----------------+

2. Log in to controller-0 and check the Pacemaker status to identify important information for the RGW migration:

   Full List of Resources:
     * ip-192.168.24.46	(ocf:heartbeat:IPaddr2):     	Started controller-0
     * ip-10.0.0.103   	(ocf:heartbeat:IPaddr2):     	Started controller-1
     * ip-172.17.1.129 	(ocf:heartbeat:IPaddr2):     	Started controller-2
     * ip-172.17.3.68  	(ocf:heartbeat:IPaddr2):     	Started controller-0
     * ip-172.17.4.37  	(ocf:heartbeat:IPaddr2):     	Started controller-1
     * Container bundle set: haproxy-bundle
   
   [undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp17-openstack-haproxy:pcmklatest]:
       * haproxy-bundle-podman-0   (ocf:heartbeat:podman):  Started controller-2
       * haproxy-bundle-podman-1   (ocf:heartbeat:podman):  Started controller-0
       * haproxy-bundle-podman-2   (ocf:heartbeat:podman):  Started controller-1

3. Identify the ranges of the storage networks. The following is an example and the values might differ in your environment:

   [heat-admin@controller-0 ~]$ ip -o -4 a
   
   1: lo	inet 127.0.0.1/8 scope host lo\   	valid_lft forever preferred_lft forever
   2: enp1s0	inet 192.168.24.45/24 brd 192.168.24.255 scope global enp1s0\   	valid_lft forever preferred_lft forever
   2: enp1s0	inet 192.168.24.46/32 brd 192.168.24.255 scope global enp1s0\   	valid_lft forever preferred_lft forever
   7: br-ex	inet 10.0.0.122/24 brd 10.0.0.255 scope global br-ex\   	valid_lft forever preferred_lft forever
   8: vlan70	inet 172.17.5.22/24 brd 172.17.5.255 scope global vlan70\   	valid_lft forever preferred_lft forever
   8: vlan70	inet 172.17.5.94/32 brd 172.17.5.255 scope global vlan70\   	valid_lft forever preferred_lft forever
   9: vlan50	inet 172.17.2.140/24 brd 172.17.2.255 scope global vlan50\   	valid_lft forever preferred_lft forever
   10: vlan30	inet 172.17.3.73/24 brd 172.17.3.255 scope global vlan30\   	valid_lft forever preferred_lft forever
   10: vlan30	inet 172.17.3.68/32 brd 172.17.3.255 scope global vlan30\   	valid_lft forever preferred_lft forever
   11: vlan20	inet 172.17.1.88/24 brd 172.17.1.255 scope global vlan20\   	valid_lft forever preferred_lft forever
   12: vlan40	inet 172.17.4.24/24 brd 172.17.4.255 scope global vlan40\   	valid_lft forever preferred_lft forever

   • br-ex represents the External Network, where in the current environment, HAProxy has the front-end Virtual IP (VIP) assigned.
   • vlan30 represents the Storage Network, where the new RGW instances should be started on the Red Hat Ceph Storage nodes.

4. Identify the network that you previously had in HAProxy and propagate it through director to the Red Hat Ceph Storage nodes. Use this network to reserve a new VIP that is owned by Red Hat Ceph Storage as the entry point for the RGW service.

   1. Log in to controller-0 and find the ceph_rgw section in the current HAProxy configuration:

      $ less /var/lib/config-data/puppet-generated/haproxy/etc/haproxy/haproxy.cfg
      ...
      ...
      listen ceph_rgw
        bind 10.0.0.103:8080 transparent
        bind 172.17.3.68:8080 transparent
        mode http
        balance leastconn
        http-request set-header X-Forwarded-Proto https if { ssl_fc }
        http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
        http-request set-header X-Forwarded-Port %[dst_port]
        option httpchk GET /swift/healthcheck
        option httplog
        option forwardfor
        server controller-0.storage.redhat.local 172.17.3.73:8080 check fall 5 inter 2000 rise 2
        server controller-1.storage.redhat.local 172.17.3.146:8080 check fall 5 inter 2000 rise 2
        server controller-2.storage.redhat.local 172.17.3.156:8080 check fall 5 inter 2000 rise 2

   2. Confirm that the network is used as an HAProxy front end. The following example shows that controller-0 exposes the services by using the external network, which is absent from the Red Hat Ceph Storage nodes. You must propagate the external network through director:

      [controller-0]$ ip -o -4 a
      
      ...
      7: br-ex	inet 10.0.0.106/24 brd 10.0.0.255 scope global br-ex\   	valid_lft forever preferred_lft forever
      ...

      Note

      If the target nodes are not managed by director, you cannot use this procedure to configure the network. An administrator must manually configure all the required networks.

5. Propagate the HAProxy front-end network to Red Hat Ceph Storage nodes.

   1. In the NIC template that you use to define the ceph-storage network interfaces, add the new config section in the Red Hat Ceph Storage network configuration template file, for example, /home/stack/composable_roles/network/nic-configs/ceph-storage.j2:

      ---
      network_config:
      - type: interface
        name: nic1
        use_dhcp: false
        dns_servers: {{ ctlplane_dns_nameservers }}
        addresses:
        - ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_cidr }}
        routes: {{ ctlplane_host_routes }}
      - type: vlan
        vlan_id: {{ storage_mgmt_vlan_id }}
        device: nic1
        addresses:
        - ip_netmask: {{ storage_mgmt_ip }}/{{ storage_mgmt_cidr }}
        routes: {{ storage_mgmt_host_routes }}
      - type: interface
        name: nic2
        use_dhcp: false
        defroute: false
      - type: vlan
        vlan_id: {{ storage_vlan_id }}
        device: nic2
        addresses:
        - ip_netmask: {{ storage_ip }}/{{ storage_cidr }}
        routes: {{ storage_host_routes }}
      - type: ovs_bridge
        name: {{ neutron_physical_bridge_name }}
        dns_servers: {{ ctlplane_dns_nameservers }}
        domain: {{ dns_search_domains }}
        use_dhcp: false
        addresses:
        - ip_netmask: {{ external_ip }}/{{ external_cidr }}
        routes: {{ external_host_routes }}
        members: []
        - type: interface
          name: nic3
          primary: true

   2. Add the External Network to the bare metal file, for example, /home/stack/composable_roles/network/baremetal_deployment.yaml that is used by metalsmith:

      Note

      Ensure that network_config_update is enabled for network propagation to the target nodes when os-net-config is triggered.

      - name: CephStorage
        count: 3
        hostname_format: cephstorage-%index%
        instances:
        - hostname: cephstorage-0
        name: ceph-0
        - hostname: cephstorage-1
        name: ceph-1
        - hostname: cephstorage-2
        name: ceph-2
        defaults:
        profile: ceph-storage
        network_config:
            template: /home/stack/composable_roles/network/nic-configs/ceph-storage.j2
            network_config_update: true
        networks:
        - network: ctlplane
            vif: true
        - network: storage
        - network: storage_mgmt
        - network: external

   3. Configure the new network on the bare metal nodes:

      (undercloud) [stack@undercloud-0]$ openstack overcloud node provision \
         -o overcloud-baremetal-deployed-0.yaml \
         --stack overcloud \
         --network-config -y \
        $PWD/composable_roles/network/baremetal_deployment.yaml

   4. Verify that the new network is configured on the Red Hat Ceph Storage nodes:

      [root@cephstorage-0 ~]# ip -o -4 a
      
      1: lo	inet 127.0.0.1/8 scope host lo\   	valid_lft forever preferred_lft forever
      2: enp1s0	inet 192.168.24.54/24 brd 192.168.24.255 scope global enp1s0\   	valid_lft forever preferred_lft forever
      11: vlan40	inet 172.17.4.43/24 brd 172.17.4.255 scope global vlan40\   	valid_lft forever preferred_lft forever
      12: vlan30	inet 172.17.3.23/24 brd 172.17.3.255 scope global vlan30\   	valid_lft forever preferred_lft forever
      14: br-ex	inet 10.0.0.133/24 brd 10.0.0.255 scope global br-ex\   	valid_lft forever preferred_lft forever

Procedure

1. Identify the Red Hat Ceph Storage nodes to deploy the new clustered NFS service, for example, cephstorage-0, cephstorage-1, cephstorage-2.

   Note

   You must deploy this service on the StorageNFS isolated network so that you can mount your existing shares through the new NFS export locations. You can deploy the new clustered NFS service on your existing CephStorage nodes or HCI nodes, or on new hardware that you enrolled in the Red Hat Ceph Storage cluster.

2. If you deployed your Red Hat Ceph Storage nodes with director, propagate the StorageNFS network to the target nodes where the ceph-nfs service is deployed.

   Note

   If the target nodes are not managed by director, you cannot use this procedure to configure the network. An administrator must manually configure all the required networks.

   1. Identify the node definition file, overcloud-baremetal-deploy.yaml, that is used in the RHOSP environment. For more information about identifying the overcloud-baremetal-deploy.yaml file, see Customizing overcloud networks in Customizing the Red Hat OpenStack Services on OpenShift deployment.

   2. Edit the networks that are associated with the Red Hat Ceph Storage nodes to include the StorageNFS network:

      - name: CephStorage
        count: 3
        hostname_format: cephstorage-%index%
        instances:
        - hostname: cephstorage-0
          name: ceph-0
        - hostname: cephstorage-1
          name: ceph-1
        - hostname: cephstorage-2
          name: ceph-2
        defaults:
          profile: ceph-storage
          network_config:
            template: /home/stack/network/nic-configs/ceph-storage.j2
            network_config_update: true
          networks:
          - network: ctlplane
            vif: true
          - network: storage
          - network: storage_mgmt
          - network: storage_nfs

   3. Edit the network configuration template file, for example, /home/stack/network/nic-configs/ceph-storage.j2, for the Red Hat Ceph Storage nodes to include an interface that connects to the StorageNFS network:

      - type: vlan
        device: nic2
        vlan_id: {{ storage_nfs_vlan_id }}
        addresses:
        - ip_netmask: {{ storage_nfs_ip }}/{{ storage_nfs_cidr }}
        routes: {{ storage_nfs_host_routes }}

   4. Update the Red Hat Ceph Storage nodes:

      $ openstack overcloud node provision \
          --stack overcloud   \
          --network-config -y  \
          -o overcloud-baremetal-deployed-storage_nfs.yaml \
          --concurrency 2 \
          /home/stack/network/baremetal_deployment.yaml

      When the update is complete, ensure that a new interface is created in theRed Hat Ceph Storage nodes and that they are tagged with the VLAN that is associated with StorageNFS.

3. Identify the IP address from the StorageNFS network to use as the Virtual IP address (VIP) for the Ceph NFS service:

   $ openstack port list -c "Fixed IP Addresses" --network storage_nfs

4. In a running cephadm shell, identify the hosts for the NFS service:

   $ ceph orch host ls

5. Label each host that you identified. Repeat this command for each host that you want to label:

   $ ceph orch host label add <hostname> nfs

   • Replace <hostname> with the name of the host that you identified.

6. Create the NFS cluster:

   $ ceph nfs cluster create cephfs \
       "label:nfs" \
       --ingress \
       --virtual-ip=<VIP> \
       --ingress-mode=haproxy-protocol

   • Replace <VIP> with the VIP for the Ceph NFS service.

     Note

     You must set the ingress-mode argument to haproxy-protocol. No other ingress-mode is supported. This ingress mode allows you to enforce client restrictions through the Shared File Systems service.

     For more information about deploying the clustered Ceph NFS service, see "Management of NFS-Ganesha gateway using the Ceph Orchestrator" in the Operations Guide for your Red Hat Ceph Storage version:

     • Red Hat Ceph Storage 7 Operations Guide
     • Red Hat Ceph Storage 8 Operations Guide
     • Red Hat Ceph Storage 9 Operations Guide

7. Check the status of the NFS cluster:

   $ ceph nfs cluster ls
   $ ceph nfs cluster info cephfs

Procedure

1. Gather the fencing information from the fencing.yaml file in your Red Hat OpenStack Platform (RHOSP) 17.1 cluster.

2. Retrieve the RHOSP 17.1 stonith configuration from any of your overcloud Controller nodes:

   $ sudo pcs config

   Stonith Devices:
   ...
     Resource: stonith-fence_ipmilan-525400dde4f7 (class=stonith
         type=fence_ipmilan)
       Attributes: stonith-fence_ipmilan-525400dde4f7-instance_attributes
         delay=20
         ipaddr=172.16.0.1
         ipport=6231
         lanplus=true
         login=admin
         passwd=password
         pcmk_host_list=compute-1
       Operations:
         monitor: stonith-fence_ipmilan-525400dde4f7-monitor-interval-60s
           interval=60s
     Resource: stonith-fence_ipmilan-525400819ad3 (class=stonith
         type=fence_ipmilan)
       Attributes: stonith-fence_ipmilan-525400819ad3-instance_attributes
         delay=20
         ipaddr=172.16.0.1
         ipport=6230
         lanplus=true
         login=admin
         passwd=password
         pcmk_host_list=compute-0
       Operations:
         monitor: stonith-fence_ipmilan-525400819ad3-monitor-interval-60s
           interval=60s
   ...

3. Generate the fencing configuration file:

   • To install the script that automatically generates this file, see How do I automatically generate fencing secret for RHOSO18 instanceha from a osp17.1 cluster that I want to adopt?.
   • To create the fencing configuration file manually, see Configuring the fencing of Compute nodes in Configuring high availability for instances.

Procedure

1. Retrieve the names of the Compute remote resources:

   $ sudo pcs stonith |grep -B1 stonith-fence_compute-fence-nova |grep Target |awk -F ': ' '{print $2}'

2. Disable the stonith and pacemaker_remote resources on each Compute remote resource:

   $ sudo pcs property set stonith-enabled=false
   $ sudo pcs resource disable <compute_remote_resource>

   where:

   <compute_remote_resource>

   Specifies the name of the Compute remote resource in your environment.

3. Retrieve the name of the Compute stonith resources:

   $ sudo pcs stonith |grep Level |grep fence_compute |awk '{print $4}' |awk -F ',' '{print $1}' |sort |uniq

4. Remove the Compute node pacemaker_remote and fencing resources:

   $ sudo pcs stonith disable stonith-fence_compute-fence-nova
   $ sudo pcs stonith disable <compute_stonith_resource>
   $ sudo pcs stonith delete <compute_stonith_resource>
   $ sudo pcs resource delete <compute_remote_resource>
   $ sudo pcs resource disable compute-unfence-trigger-clone
   $ sudo pcs resource delete compute-unfence-trigger-clone
   $ sudo pcs resource disable nova-evacuate
   $ sudo pcs resource delete nova-evacuate

   where:

   <compute_stonith_resource>

   Specifies the name of the Compute stonith resource in your environment.