5.6. Updating the control plane for new edge sites
Update the control plane configuration to integrate newly deployed edge locations. This enables the control plane to manage storage and compute services at the new edge sites.
You can create additional node sets by using the OpenStackDataPlaneNodeSet custom resource (CR). Use a unique availability zone, and the VLANs, NIC mappings, and IP addresses specific to the site you are deploying. For more information about deploying an OpenStackDataPlaneNodeSet CR, see Creating the data plane.
When you deploy a DCN node set with storage, you must update two fields of the OpenStackControlPlane CR at the central location:
-
cinderVolumes -
glanceAPIs - Neutron
- OVN
If you are using cells, you must also configure cells for the new DCN site.
Prerequisites
- You have deployed the central location.
-
You have deployed an additional
OpenStackDataPlanenode set.
Procedure
In the neutron service configuration, update the customServiceConfig field to add the new availability zone and network leaf:
customServiceConfig: | [DEFAULT] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.AZLeastRoutersScheduler network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.AZAwareWeightScheduler default_availability_zones = az0,az1 [ml2_type_vlan] network_vlan_ranges = datacentre:1:1000,leaf1:1:1000 [neutron] physnets = datacentre,leafIn the OVN service configuration, update the availability zones:
ovnController: external-ids: availability-zones: - az0 - az1 enable-chassis-as-gateway: true ovn-bridge: br-int ovn-encap-type: geneve system-id: random networkAttachment: tenant nicMappings: datacentre: ospbrUpdate the
cinderVolumesfield in theOpenStackControlPlaneCR to add the availability zones definitions of the remote location. Each cinder volume service in each availability zone uses the Glance API server for its availability zone. For example,glance_api_servers = https://glance-az1-internal.openstack.svc:9292:cinderVolumes: az0: customServiceConfig: | [DEFAULT] .... az1: customServiceConfig: | [DEFAULT] enabled_backends = ceph glance_api_servers = https://glance-az1-internal.openstack.svc:9292 [ceph] volume_backend_name = az1 volume_driver = cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf = /etc/ceph/az1.conf rbd_user = openstack rbd_pool = volumes rbd_flatten_volume_from_snapshot = False rbd_secret_uuid = 19ccdd60-79a0-5f0f-aece-ece700e514f8 rbd_cluster_name = az1 backend_availability_zone = az1Register an Image service (glance) pod to the Identity service (keystone) catalog:
In DCN, an Image service pod is deployed for each node set. A single Image service pod is registered to the Identity service catalog at any one time. For this reason, in the top-level Glance custom resource (CR), the
keystoneEndpointparameter is defined and exposed. Unless a single instance is deployed, the human operator can choose, before the mainOpenStackControlPlaneCR is applied, which instance should be registered. Because our default endpoint is theaz0Image service API, thekeystoneEndpointis set toaz0:spec: <...> glance: enabled: true keystoneEndpoint: az0 glanceAPIs: az0: apiTimeout: 60Update the
glanceAPIsfield:For the node sets at
az0, theglanceAPIsfield configures the Image service pods for the central location. When you add an additional node set in AZ1, theOpenStackControlPlaneCR is updated such that theglanceAPIsfield contains the Image service (glance) pod definition for AZ0 and AZ1. Additionally, the Image service pod for AZ1 defines the ceph the backend for the central location, and the AZ0 Image service pod for the central location is updated so that it has the ceph backend definition for AZ1.glanceAPIs: az1: customServiceConfig: | [DEFAULT] enabled_import_methods = [web-download,copy-image,glance-direct] enabled_backends = az0:rbd,az1:rbd [glance_store] default_backend = az1 [az1] rbd_store_ceph_conf = /etc/ceph/az1.conf store_description = "az1 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az0] rbd_store_ceph_conf = /etc/ceph/az0.conf store_description = "az0 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True networkAttachments: - storage override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.81 spec: type: LoadBalancer replicas: 2 type: edge az0: customServiceConfig: | [DEFAULT] enabled_import_methods = [web-download,copy-image,glance-direct] enabled_backends = az0:rbd,az1:rbd [glance_store] default_backend = az0 [az0] rbd_store_ceph_conf = /etc/ceph/az0.conf store_description = "az0 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az1] rbd_store_ceph_conf = /etc/ceph/az1.conf store_description = "az1 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True networkAttachments: - storage override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer replicas: 3 type: split참고Availability zone
az0is of typesplit, and all other availability zones are of typeedge.The split type is for cloud users to use when uploading images. The edge type is created so that when Cinder or Nova interact with Glance they can be configured to whichever glance is local to them. Use at least 3 replicas for the default split glance pods and 2 replicas for the edge glance pods and increase replicas proportionally to the workload.
Optional: Update the Cell configuration.
By default, compute nodes across all availability zones (AZs) are placed in a common cell called
cell1. You can increase the performance of large deployments by partitioning compute nodes into separate cells. For DCN deployments, place each availability zone into its own cell. For more information, see Adding Compute cells to the control plane.Apply the changes made to the
OpenStackControlPlaneCR:oc apply -f openstack_control_plane.yamlContinue to update the control plane for each additional edge site that is added. Add Red Hat Storage Service (RHCS) to your OpenShift secrets as needed.
In the neutron service configuration, update the customServiceConfig field to add the new availability zone and network leaf:
customServiceConfig: | [DEFAULT] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.AZLeastRoutersScheduler network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.AZAwareWeightScheduler default_availability_zones = az0,az1,az2 [ml2_type_vlan] network_vlan_ranges = datacentre:1:1000,leaf1:1:1000,leaf2:1:1000 [neutron] physnets = datacentre,leaf1,leaf2In the OVN service configuration, update the availability zones
ovnController: external-ids: availability-zones: - az0 - az1 - az2 enable-chassis-as-gateway: true ovn-bridge: br-int ovn-encap-type: geneve system-id: random networkAttachment: tenant nicMappings: datacentre: ospbrAdd an additional availability zone the
cinderVolumesservice field:cinderVolumes: az0: customServiceConfig: | [DEFAULT] ... az1: customServiceConfig: | [DEFAULT] ... az2: customServiceConfig: | [DEFAULT] enabled_backends = ceph glance_api_servers = https://glance-az2-internal.openstack.svc:9292 [ceph] volume_backend_name = ceph volume_driver = cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf = /etc/ceph/az2.conf rbd_user = openstack rbd_pool = volumes rbd_flatten_volume_from_snapshot = False rbd_secret_uuid = 5c0c7a8e-55b1-5fa8-bc5c-9756b7862d2f rbd_cluster_name = az2 backend_availability_zone = az2Add an additional availability zone to the
glanceAPIsfield:As you add additional AZs, you must ensure that each Image service pod definition contains the storage configuration of the central location (AZ0), and its own local ceph configuration. You must also ensure that the central location has the storage definition of all other sites. This creates a hub and spoke relationship between the central location Image service pod, and the Image service pods for geographically dispersed node sets:
glanceAPIs: az0: customServiceConfig: | [DEFAULT] enabled_import_methods = [web-download,copy-image,glance-direct] enabled_backends = az0:rbd,az1:rbd,az2:rbd [glance_store] default_backend = az0 [az0] rbd_store_ceph_conf = /etc/ceph/az0.conf store_description = "az0 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az1] rbd_store_ceph_conf = /etc/ceph/az1.conf store_description = "az1 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az2] rbd_store_ceph_conf = /etc/ceph/az2.conf store_description = "az2 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True networkAttachments: - storage override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.80 spec: type: LoadBalancer replicas: 3 type: split az1: customServiceConfig: | [DEFAULT] enabled_import_methods = [web-download,copy-image,glance-direct] enabled_backends = az0:rbd,az1:rbd [glance_store] default_backend = az1 [az1] rbd_store_ceph_conf = /etc/ceph/az1.conf store_description = "az1 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az0] rbd_store_ceph_conf = /etc/ceph/az0.conf store_description = "az0 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True networkAttachments: - storage override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.81 spec: type: LoadBalancer replicas: 2 type: edge az2: customServiceConfig: | [DEFAULT] enabled_import_methods = [web-download,copy-image,glance-direct] enabled_backends = az0:rbd,az2:rbd [glance_store] default_backend = az2 [az2] rbd_store_ceph_conf = /etc/ceph/az2.conf store_description = "az2 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True [az0] rbd_store_ceph_conf = /etc/ceph/az0.conf store_description = "az0 RBD backend" rbd_store_pool = images rbd_store_user = openstack rbd_thin_provisioning = True networkAttachments: - storage override: service: internal: metadata: annotations: metallb.universe.tf/address-pool: internalapi metallb.universe.tf/allow-shared-ip: internalapi metallb.universe.tf/loadBalancerIPs: 172.17.0.82 spec: type: LoadBalancer replicas: 2 type: edge
Apply the changes made to the
OpenStackControlPlaneCR:oc apply -f openstack_control_plane.yamlAdd the AZ to a host aggregate. This allows OpenStack administrators to schedule workloads to a geographical location by passing the
--availability-zoneargument:Open a terminal to the
openstackclientpod:# oc rsh openstackclientCreate a new OpenStack aggregate:
$ openstack aggregate create <aggregate_name>Label the OpenStack aggregate with the name of the AZ:
$ openstack aggregate set --zone <availability_zone> <aggregate_name>Add each host in the AZ to the aggregate:
$ openstack aggregate add host <aggregate_name> <compute_node_1> $ openstack aggregate add host <aggregate_name> <compute_node_2> ...