Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Configuring the overcloud

Use Red Hat OpenStack Platform (RHOSP) director to install and configure spine leaf networking in the RHOSP overcloud. The high-level steps are:

  1. Define the overcloud networks for each leaf.
  2. Create a composable role for each leaf and attach the composable network to each respective role.
  3. Create a unique NIC configuration for each role.
  4. Set the control plane parameters and the change bridge mappings so that each leaf routes traffic through the specific bridge or VLAN on that leaf.
  5. Define virtual IPs (VIPs) for your overcloud endpoints, and identify the subnet for each VIP.
  6. Provision your overcloud networks and overcloud VIPs.

  7. Register the bare metal nodes in your overcloud.

    Note

    Skip steps 7, 8, and 9 if you are using pre-provisioned bare metal nodes.

  8. Introspect the bare metal nodes in your overcloud.
  9. Provision bare metal nodes.
  10. Deploy your overcloud using the configuration you set in the earlier steps.

4.1. Defining the leaf networks

The Red Hat OpenStack Platform (RHOSP) director creates the overcloud leaf networks from a YAML-formatted, custom network definition file that you construct. This custom network definition file lists each composable network and its attributes and also defines the subnets needed for each leaf.

Complete the following steps to create a YAML-formatted, custom network definition file that contains the specifications for your spine-leaf network on the overcloud. Later, the provisioning process creates a heat environment file from your network definition file that you will include when you deploy your RHOSP overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Create a templates directory under /home/stack: 

    $ mkdir /home/stack/templates

  4. Use the default template, routed-networks.yaml, template as a basis to create a custom network definition template for your environment, by copying it to your templates directory:

    Example

    $ cp /usr/share/openstack-tripleo-heat-templates/network-data-samples/\
routed-networks.yaml \
/home/stack/templates/spine-leaf-networks-data.yaml

  5. Edit your copy of the network definition template to define each base network and respective leaf subnets as a composable network item.

    Tip

    For information, see Network definition file configuration options in the Director Installation and Usage guide.

    Example

    The following example demonstrates how to define the Internal API network and its leaf networks: 

    - name: InternalApi
  name_lower: internal_api
  vip: true
  mtu: 1500
  subnets:
    internal_api_subnet:
      ip_subnet: 172.16.32.0/24
      gateway_ip: 172.16.32.1
      allocation_pools:
        - start: 172.16.32.4
          end: 172.16.32.250
      vlan: 20
    internal_api_leaf1_subnet:
      ip_subnet: 172.16.33.0/24
      gateway_ip: 172.16.33.1
      allocation_pools:
        - start: 172.16.33.4
          end: 172.16.33.250
      vlan: 30
    internal_api_leaf2_subnet:
      ip_subnet: 172.16.34.0/24
      gateway_ip: 172.16.34.1
      allocation_pools:
        - start: 172.16.34.4
          end: 172.16.34.250
      vlan: 40
Note

You do not define the Control Plane networks in your custom network definition template because the undercloud has already created these networks. However, you must set the parameters manually so that the overcloud can configure the NICs accordingly. For more information, see Configuring routed spine-leaf in the undercloud.

Note

There is currently no automatic validation for the network subnet and allocation_pools values. Ensure that you define these values consistently and that there is no conflict with existing networks.

Note

Add the vip parameter and set the value to true for the networks that host the Controller-based services. In this example, the InternalApi network contains these services.

Next steps

  1. Note the path and file name of the custom network definition file that you have created. You will need this information later when you provision your networks for the RHOSP overcloud.
  2. Proceed to the next step Defining leaf roles and attaching networks.

Additional resources

4.2. Defining leaf roles and attaching networks

The Red Hat OpenStack Platform (RHOSP) director creates a composable role for each leaf and attaches the composable network to each respective role from a roles template that you construct. Start by copying the default Controller, Compute, and Ceph Storage roles from the director core templates, and modifying these to meet your environment’s needs. After you have created all of the individual roles, you run the openstack overcloud roles generate command to concatenate them into one large custom roles data file.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Copy the default roles for Controller, Compute, and Ceph Storage roles that ship with RHOSP to the home directory of the stack user. Rename the files to reflect that they are leaf 0: 

    $ cp /usr/share/openstack-tripleo-heat-templates/roles/Controller.yaml ~/roles/Controller0.yaml
$ cp /usr/share/openstack-tripleo-heat-templates/roles/Compute.yaml ~/roles/Compute0.yaml
$ cp /usr/share/openstack-tripleo-heat-templates/roles/CephStorage.yaml ~/roles/CephStorage0.yaml

  4. Copy the leaf 0 files as a basis for your leaf 1 and leaf 2 files: 

    $ cp ~/roles/Compute0.yaml ~/roles/Compute1.yaml
$ cp ~/roles/Compute0.yaml ~/roles/Compute2.yaml
$ cp ~/roles/CephStorage0.yaml ~/roles/CephStorage1.yaml
$ cp ~/roles/CephStorage0.yaml ~/roles/CephStorage2.yaml

  5. Edit the parameters in each file to align with their respective leaf parameters.

    Tip

    For information about the various parameters in a roles data template, see Examining role parameters in the Director Installation and Usage guide.

    Example - ComputeLeaf0

    - name: ComputeLeaf0
  HostnameFormatDefault: '%stackname%-compute-leaf0-%index%'

    Example - CephStorageLeaf0

    - name: CephStorageLeaf0
  HostnameFormatDefault: '%stackname%-cephstorage-leaf0-%index%'

  6. Edit the network parameter in the leaf 1 and leaf 2 files so that they align with the respective leaf network parameters.

    Example - ComputeLeaf1

    - name: ComputeLeaf1
  networks:
    InternalApi:
      subnet: internal_api_leaf1
    Tenant:
      subnet: tenant_leaf1
    Storage:
      subnet: storage_leaf1

    Example - CephStorageLeaf1

    - name: CephStorageLeaf1
  networks:
    Storage:
      subnet: storage_leaf1
    StorageMgmt:
      subnet: storage_mgmt_leaf1

    Note

    This applies only to leaf 1 and leaf 2. The network parameter for leaf 0 retains the base subnet values, which are the lowercase names of each subnet combined with a _subnet suffix. For example, the Internal API for leaf 0 is internal_api_subnet.

  7. When your role configuration is complete, run the overcloud roles generate command to generate the full roles data file.

    Example

    $ openstack overcloud roles generate --roles-path ~/roles -o spine-leaf-roles-data.yaml Controller Compute Compute1 Compute2 CephStorage CephStorage1 CephStorage2

    This creates one custom roles data file that includes all of the custom roles for each respective leaf network.

Next steps

  1. Note the path and file name of the custom roles data file that the overcloud roles generate command has output. You will need this information later when you deploy your overcloud.
  2. Proceed to the next step Creating a custom NIC configuration for leaf roles.

Additional resources

4.3. Creating a custom NIC configuration for leaf roles

Each role that the Red Hat OpenStack Platform (RHOSP) director creates requires a unique NIC configuration. Complete the following steps to create a custom set of NIC templates and a custom environment file that maps the custom templates to the respective role.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.
  • You have a custom network definition file.
  • You have a custom roles data file.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Copy the content from one of the default NIC templates to use as a basis for a custom template for your NIC configuration.

    Example

    In this example, the single-nic-vlans NIC template is being copied and will be used as the basis for a custom template for your NIC configuration: 

    $ cp -r /usr/share/ansible/roles/tripleo_network_config/\
templates/single-nic-vlans/* /home/stack/templates/spine-leaf-nics/.

  4. Edit each NIC configuration in the NIC templates that you copied in the earlier step to reflect the specifics for your spine-leaf topology.

    Example

    {% set mtu_list = [ctlplane_mtu] %}
{% for network in role_networks %}
{{ mtu_list.append(lookup('vars', networks_lower[network] ~ '_mtu')) }}
{%- endfor %}
{% set min_viable_mtu = mtu_list | max %}
network_config:
- type: ovs_bridge
  name: {{ neutron_physical_bridge_name }}
  mtu: {{ min_viable_mtu }}
  use_dhcp: false
  dns_servers: {{ ctlplane_dns_nameservers }}
  domain: {{ dns_search_domains }}
  addresses:
  - ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_subnet_cidr }}
  routes: {{ ctlplane_host_routes }}
  members:
  - type: interface
    name: nic1
    mtu: {{ min_viable_mtu }}
    # force the MAC address of the bridge to this interface
    primary: true
{% for network in role_networks %}
  - type: vlan
    mtu: {{ lookup('vars', networks_lower[network] ~ '_mtu') }}
    vlan_id: {{ lookup('vars', networks_lower[network] ~ '_vlan_id') }}
    addresses:
    - ip_netmask:
        {{ lookup('vars', networks_lower[network] ~ '_ip') }}/{{ lookup('vars', networks_lower[network] ~ '_cidr') }}
    routes: {{ lookup('vars', networks_lower[network] ~ '_host_routes') }}
{% endfor %}

    Tip

    For more information, see Custom network interface templates in the Director Installation and Usage guide.

  5. Create a custom environment file, such as spine-leaf-nic-roles-map.yaml, that contains a parameter_defaults section that maps the custom NIC templates to each custom role. 

    parameter_defaults:
  %%ROLE%%NetworkConfigTemplate: <path_to_ansible_jinja2_nic_config_file>

    Example

    parameter_defaults:
  Controller0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  Controller1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  Controller2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'

Next steps

  1. Note the path and file name of your custom NIC templates and the custom environment file that maps the custom NIC templates to each custom role. You will need this information later when you deploy your overcloud.
  2. Proceed to the next step Mapping separate networks and setting control plane parameters.

Additional resources

4.4. Mapping separate networks and setting control plane parameters

In a spine leaf architecture, each leaf routes traffic through the specific bridge or VLAN on that leaf, which is often the case with edge computing scenarios. So, you must change the default mappings where the Red Hat OpenStack Platform (RHOSP) Controller and Compute network configurations use a br-ex bridge.

The RHOSP director creates the control plane network during undercloud creation. However, the overcloud requires access to the control plane for each leaf. To enable this access, you must define additional parameters in your deployment.

Complete the following steps to create a custom network environment file that contains the separate network mappings and sets access to the control plane networks for the overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. In a new custom environment file, such as spine-leaf-ctlplane.yaml, create a parameter_defaults section and set the NeutronBridgeMappings parameter for each leaf that uses the default br-ex bridge.

    Important

    The name of the custom environment file that you create to contain your network definition must end in either .yaml or .template.

    • For flat network mappings, list each leaf in the NeutronFlatNetworks parameter and set the NeutronBridgeMappings parameter for each leaf:

      Example

      parameter_defaults:
  NeutronFlatNetworks: leaf0,leaf1,leaf2
  Controller0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Controller1Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Controller2Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Compute0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Compute1Parameters:
    NeutronBridgeMappings: "leaf1:br-ex"

  Compute2Parameters:
    NeutronBridgeMappings: "leaf2:br-ex"

      Tip

      For more information, see Chapter 17. Networking (neutron) Parameters in the Overcloud Parameters guide

    • For VLAN network mappings, add vlan to NeutronNetworkType, and by using NeutronNetworkVLANRanges, map VLANs for the leaf networks:

      Example

      parameter_defaults:
  NeutronNetworkType: 'geneve,vlan'
  NeutronNetworkVLANRanges: 'leaf0:1:1000,leaf1:1:1000,leaf2:1:1000'

  Controller0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Controller1Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Controller2Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Compute0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"

  Compute1Parameters:
    NeutronBridgeMappings: "leaf1:br-ex"

  Compute2Parameters:
    NeutronBridgeMappings: "leaf2:br-ex"

      Note

      You can use both flat networks and VLANs in your spine-leaf topology.

  4. Add the control plane subnet mapping for each spine-leaf network by using the <role>ControlPlaneSubnet parameter:

    Example

    parameter_defaults:
  NeutronFlatNetworks: leaf0,leaf1,leaf2
  Controller0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
    ControllerControlPlaneSubnet: leaf0
  Controller1Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
    Controller1ControlPlaneSubnet: leaf0
  Controller2Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
    Controller2ControlPlaneSubnet: leaf0
  Compute0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
    Compute0ControlPlaneSubnet: leaf0
  CephStorage0Parameters:
    CephStorage0ControlPlaneSubnet: leaf0
  Compute1Parameters:
    NeutronBridgeMappings: "leaf1:br-ex"
    Compute1ControlPlaneSubnet: leaf1
  CephStorage1Parameters:
    CephStorage1ControlPlaneSubnet: leaf1
  Compute2Parameters:
    NeutronBridgeMappings: "leaf2:br-ex"
    Compute2ControlPlaneSubnet: leaf2
  CephStorage2Parameters:
    CephStorage2ControlPlaneSubnet: leaf2

Next steps

  1. Note the path and file name of the custom network environment file that you have created. You will need this information later when you deploy your overcloud.
  2. Proceed to the next step Setting the subnet for virtual IP addresses.

Additional resources

4.5. Setting the subnet for virtual IP addresses

The Red Hat OpenStack Platform (RHOSP) Controller role typically hosts virtual IP (VIP) addresses for each network. By default, the RHOSP overcloud takes the VIPs from the base subnet of each network except for the control plane. The control plane uses ctlplane-subnet, which is the default subnet name created during a standard undercloud installation.

In this spine-leaf scenario, the default base provisioning network is leaf0 instead of ctlplane-subnet. This means that you must add overriding values to the VipSubnetMap parameter to change the subnet that the control plane VIP uses.

Additionally, if the VIPs for each network do not use the base subnet of one or more networks, you must add additional overrides to the VipSubnetMap parameter to ensure that the RHOSP director creates VIPs on the subnet associated with the L2 network segment that connects the Controller nodes.

Complete the following steps to create a YAML-formatted, custom network VIP definition file that contains the overrides for your VIPs on the overcloud. Later, the provisioning process creates a heat environment file from your network VIP definition file that you will include when you deploy your RHOSP overcloud. You will also use your network VIP definition file when you run the overcloud deploy command.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. In a new custom network VIP definition template, such as spine-leaf-vip-data.yaml, create a parameter_defaults section and add the VipSubnetMap parameter based on your requirements.

    • If you use leaf0 for the provisioning-control plane network, set the ctlplane VIP remapping to leaf0: 

      parameter_defaults:
  VipSubnetMap:
    ctlplane: leaf0
      Tip

      For more information, see Configuring and provisioning network VIPs for the overcloud in the Director Installation and Usage guide.

    • If you use a different leaf for multiple VIPs, set the VIP remapping to suit these requirements. For example, use the following snippet to configure the VipSubnetMap parameter to use leaf1 for all VIPs: 

      parameter_defaults:
  VipSubnetMap:
    ctlplane: leaf1
    redis: internal_api_leaf1
    InternalApi: internal_api_leaf1
    Storage: storage_leaf1
    StorageMgmt: storage_mgmt_leaf1

Next steps

  1. Note the path and file name of the custom network VIP definition template that you have created. You will need this information later when you provision your network VIPs for the RHOSP overcloud.
  2. Proceed to the next step Provisioning networks and VIPs for the overcloud.

Additional resources

4.6. Provisioning networks and VIPs for the overcloud

The Red Hat OpenStack Platform (RHOSP) provisioning process creates a heat environment file from your network definition file that contains your network specifications. If you are using VIPs, the RHOSP provisioning process works the same way: RHOSP creates a heat environment file from your VIP definition file that contains your VIP specifications. After you provision your networks and VIPs, you have two heat environment files that you will use later to deploy your overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.
  • You have a network configuration template.
  • If you are using VIPs, you have a VIP definition template.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Using the network configuration template that was created earlier, provision your overcloud networks, using the --output option to name the file that the overcloud network provision command outputs:

    Tip

    For more information, see Configuring and provisioning overcloud network definitions in the Director Installation and Usage guide.

    Example

    $ openstack overcloud network provision \
  --output spine-leaf-networks-provisioned.yaml \
 /home/stack/templates/spine_leaf_networks_data.yaml

    Important

    The name of the output file that you specify must end in either .yaml or .template.

  4. Using the VIP definition file created earlier, provision your overcloud VIPs, using the --output option to name the file that the overcloud network provision command outputs:

    Tip

    For more information, see Configuring and provisioning network VIPs for the overcloud in the Director Installation and Usage guide. 

    $ openstack overcloud network vip provision \
  --stack spine_leaf_overcloud \
 --output spine-leaf-vips_provisioned.yaml \
 /home/stack/templates/spine_leaf_vip_data.yaml
    Important

    The name of the output file that you specify must end in either .yaml or .template.

  5. Note the path and file names of the generated output files. You will need this information later when you deploy your overcloud.

Verification

  • You can use the following commands to confirm that the command created your overcloud networks and subnets: 

    $ openstack network list
$ openstack subnet list
$ openstack network show <network>
$ openstack subnet show <subnet>
$ openstack port list
$ openstack port show <port>

    Replace <network>, <subnet>, and <port> with the name or UUID of the network, subnet, and port that you want to check.

Next steps

  1. If you are using pre-provisioned nodes, skip to Running the overcloud deployment command.
  2. Otherwise, proceed to the next step Registering bare metal nodes on the overcloud.

Additional resources

4.7. Registering bare metal nodes on the overcloud

Registering your physical machines is the first of three steps for provisioning bare metal nodes. Red Hat OpenStack Platform (RHOSP) director requires a custom node definition template that specifies the hardware and power management details of your physical machines. You can create this template in JSON or YAML formats. After you register your physical machines as bare metal nodes, you introspect them, and then you finally provision them.

Note

If you are using pre-provisioned bare metal nodes then you can skip registering and introspecting bare metal nodes on the overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Inside a new node definition template, such as barematal-nodes.yaml, create a list of your physical machines that specifies their hardware and power management details.

    Example

    nodes:
  - name: "node01"
    ports:
      - address: "aa:aa:aa:aa:aa:aa"
        physical_network: ctlplane
        local_link_connection:
          switch_id: 52:54:00:00:00:00
          port_id: p0
    cpu: 4
    memory: 6144
    disk: 40
    arch: "x86_64"
    pm_type: "ipmi"
    pm_user: "admin"
    pm_password: "p@55w0rd!"
    pm_addr: "192.168.24.205"
  - name: "node02"
    ports:
      - address: "bb:bb:bb:bb:bb:bb"
        physical_network: ctlplane
        local_link_connection:
          switch_id: 52:54:00:00:00:00
          port_id: p0
    cpu: 4
    memory: 6144
    disk: 40
    arch: "x86_64"
    pm_type: "ipmi"
    pm_user: "admin"
    pm_password: "p@55w0rd!"
    pm_addr: "192.168.24.206"

    Tip

    For more information about template parameter values and for a JSON example, see Registering nodes for the overcloud in the Director Installation and Usage guide.

  4. Verify the template formatting and syntax.

    Example

    $ openstack overcloud node import --validate-only ~/templates/\
baremetal-nodes.yaml

  5. Correct any errors and save your node definition template.

  6. Import your node definition template to RHOSP director to register each node from your template into director:

    Example

    $ openstack overcloud node import ~/baremetal-nodes.yaml

Verification

  • When the node registration and configuration is complete, confirm that director has successfully registered the nodes: 

    $ openstack baremetal node list

    The baremetal node list command should include the imported nodes and the status should be manageable.

Next steps

Additional resources

4.8. Introspecting bare metal nodes on the overcloud

After you register a physical machine as a bare metal node, you can automatically add its hardware details and create ports for each of its Ethernet MAC addresses by using Red Hat OpenStack Platform (RHOSP) director introspection. After you perform introspection on your bare metal nodes, the final step is to provision them.

Note

If you are using pre-provisioned bare metal nodes then you can skip registering and introspecting bare metal nodes on the overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.
  • You have registered your bare metal nodes for your overcloud with RHOSP.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the undercloud credentials file: 

    $ source ~/stackrc

  3. Run the pre-introspection validation group to check the introspection requirements: 

    $ validation run --group pre-introspection
  4. Review the results of the validation report.

  5. (Optional) Review detailed output from a specific validation: 

    $ validation history get --full <UUID>

    Replace <UUID> with the UUID of the specific validation from the report that you want to review.

    Important

    A FAILED validation does not prevent you from deploying or running RHOSP. However, a FAILED validation can indicate a potential issue with a production environment.

  6. Inspect the hardware attributes of all nodes: 

    $ openstack overcloud node introspect --all-manageable --provide
    Tip

    For more information, see Using director introspection to collect bare metal node hardware information in the Director Installation and Usage guide.

    Monitor the introspection progress logs in a separate terminal window: 

    $ sudo tail -f /var/log/containers/ironic-inspector/ironic-inspector.log

Verification

  • After the introspection completes, all nodes change to an available state.

Next steps

Additional resources

4.9. Provisioning bare metal nodes for the overcloud

To provision your bare metal nodes for Red Hat OpenStack Platform (RHOSP), you define the quantity and attributes of the bare metal nodes that you want to deploy in a node definition file in YAML format, and assign overcloud roles to these nodes. You also define the network layout of the nodes.

The provisioning process creates a heat environment file from your node definition file. This heat environment file contains the node specifications you configured in your node definition file, including node count, predictive node placement, custom images, and custom NICs. When you deploy your overcloud, include this file in the deployment command. The provisioning process also provisions the port resources for all networks defined for each node or role in the node definition file.

Note

If you are using pre-provisioned bare metal nodes then you can skip provisioning bare metal nodes on the overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.
  • The bare metal nodes are registered, introspected, and available for provisioning and deployment.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Create a bare metal nodes definition file, such as spine-leaf-baremetal-nodes.yaml, and define the node count for each role that you want to provision.

    Example

    - name: Controller
  count: 3
  defaults:
    networks:
    - network: ctlplane
      vif: true
    - network: external
      subnet: external_subnet
    - network: internal_api
      subnet: internal_api_subnet01
    - network: storage
      subnet: storage_subnet01
    - network: storage_mgmt
      subnet: storage_mgmt_subnet01
    - network: tenant
      subnet: tenant_subnet01
    network_config:
      template: /home/stack/templates/spine-leaf-nics/single-nic-vlans.j2
      default_route_network:
      - external
- name: Compute0
  count: 1
  defaults:
    networks:
    - network: ctlplane
      vif: true
    - network: internal_api
      subnet: internal_api_subnet02
    - network: tenant
      subnet: tenant_subnet02
    - network: storage
      subnet: storage_subnet02
    network_config:
      template: /home/stack/templates/spine-leaf-nics/single-nic-vlans.j2
- name: Compute1
...

    Tip

    For more information about the properties that you can set bare metal node definition file, see Provisioning bare metal nodes for the overcloud in the Director Installation and Usage guide.

  4. Provision the overcloud bare metal nodes, using the overcloud node provision command.

    Example

    $ openstack overcloud node provision \
 --stack spine_leaf_overcloud \
 --network-config \
 --output spine-leaf-baremetal-nodes-provisioned.yaml \
 /home/stack/templates/spine-leaf-baremetal-nodes.yaml

    Important

    The name of the output file that you specify must end in either .yaml or .template.

  5. Monitor the provisioning progress in a separate terminal. When provisioning is successful, the node state changes from available to active: 

    $ watch openstack baremetal node list

  6. Use the metalsmith tool to obtain a unified view of your nodes, including allocations and ports: 

    $ metalsmith list
  7. Note the path and file name of the generated output file. You will need this information later when you deploy your overcloud.

Verification

  • Confirm the association of nodes to hostnames: 

    $ openstack baremetal allocation list

Next steps

Additional resources

4.10. Deploying a spine-leaf enabled overcloud

The last step in deploying your Red Hat OpenStack Platform (RHOSP) overcloud is to run the overcloud deploy command. This command uses as inputs all of the various overcloud templates and environment files that you have constructed that represents the blueprint of your overcloud. Using these templates and environment files, the RHOSP director installs and configures your overcloud.

Prerequisites

  • Access to the undercloud host and credentials for the stack user.
  • You have performed all of the steps listed in the earlier procedures in this section and have assembled all of the various heat templates and environment files to use as inputs for the overcloud deploy command.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Collate the custom environment files and custom templates that you need for your overcloud environment, both the unedited heat template files provided with your director installation, and the custom files you created. This should include the following files:

  4. Enter the overcloud deploy command by carefully ordering the custom environment files and custom templates that are inputs to the command.

    The general rule is to specify any unedited heat template files first, followed by your custom environment files and custom templates that contain custom configurations, such as overrides to the default properties.

    In particular, follow this order for listing the inputs to the overcloud deploy command:

    1. Include your custom environment file that contains your custom NIC templates mapped to each role, for example, spine-leaf-nic-roles-map.yaml, after network-environment.yaml.

      The network-environment.yaml file provides the default network configuration for composable network parameters, that your mapping file overrides. Note that the director renders this file from the network-environment.j2.yaml Jinja2 template.

    2. If you created any other spine leaf network environment files, include these environment files after the roles-NIC templates mapping file.

    3. Add any additional environment files. For example, an environment file with your container image locations or Ceph cluster configuration.

      Example

      The following command snippet demonstrates the ordering: 

      $ openstack overcloud deploy --templates \
  -n /home/stack/templates/spine-leaf-networks-data.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml \
  -e /home/stack/templates/spine-leaf-nic-roles-map.yaml \
  -e /home/stack/templates/spine-leaf-ctlplane.yaml \
  -e /home/stack/templates/spine-leaf-vip-data.yaml \
  -e /home/stack/templates/spine-leaf-baremetal-provisioned.yaml \
  -e /home/stack/templates/spine-leaf-networks-provisioned.yaml \
  -e /home/stack/templates/spine-leaf-vips-provisioned.yaml \
  -e /home/stack/containers-prepare-parameter.yaml \
  -e /home/stack/inject-trust-anchor-hiera.yaml \
  -r /home/stack/templates/spine-leaf-roles-data.yaml
      Tip

      For more information, see Creating your overcloud in the Director Installation and Usage guide.

  5. Run the overcloud deploy command.

    When the overcloud creation completes, director provides details to access your overcloud.

Verification

Additional resources

4.11. Adding a new leaf to a spine-leaf deployment

When increasing network capacity or adding a new physical site, you might need to add a new leaf to your Red Hat OpenStack Platform (RHOSP) spine-leaf network.

Prerequisites

  • Your RHOSP deployment uses a spine-leaf network topology.

Procedure

  1. Log in to the undercloud host as the stack user.

  2. Source the stackrc undercloud credential file: 

    $ source ~/stackrc

  3. Open your network definition template, for example, /home/stack/templates/spine-leaf-networks-data.yaml. Under the appropriate base network, add a leaf subnet as a composable network item for the new leaf that you are adding.

    Example

    In this example, a subnet entry for the new leaf (leaf3) has been added: 

    - name: InternalApi
  name_lower: internal_api
  vip: true
  vlan: 10
  ip_subnet: '172.18.0.0/24'
  allocation_pools: [{'start': '172.18.0.4', 'end': '172.18.0.250'}]
  gateway_ip: '172.18.0.1'
  subnets:
    internal_api_leaf1:
      vlan: 11
      ip_subnet: '172.18.1.0/24'
      allocation_pools: [{'start': '172.18.1.4', 'end': '172.18.1.250'}]
      gateway_ip: '172.18.1.1'
    internal_api_leaf2:
      vlan: 12
      ip_subnet: '172.18.2.0/24'
      allocation_pools: [{'start': '172.18.2.4', 'end': '172.18.2.250'}]
      gateway_ip: '172.18.2.1'
    internal_api_leaf3:
      vlan: 13
      ip_subnet: '172.18.3.0/24'
      allocation_pools: [{'start': '172.18.3.4', 'end': '172.18.3.250'}]
      gateway_ip: '172.18.3.1'

  4. Create a roles data file for the new leaf that you are adding.

    1. Copy a leaf Compute and a leaf Ceph Storage file for the new leaf that you are adding.

      Example

      In this example, Compute1.yaml and CephStorage1.yaml are copied for the new leaf, Compute3.yaml and CephStorage3.yaml, respectively: 

      $ cp ~/roles/Compute1.yaml ~/roles/Compute3.yaml
$ cp ~/roles/CephStorage1.yaml ~/roles/CephStorage3.yaml

    2. Edit the name and HostnameFormatDefault parameters in the new leaf files so that they align with the respective leaf parameters.

      Example

      For example, the parameters in the Leaf 1 Compute file have the following values: 

      - name: ComputeLeaf1
  HostnameFormatDefault: '%stackname%-compute-leaf1-%index%'

      Example

      The Leaf 1 Ceph Storage parameters have the following values: 

      - name: CephStorageLeaf1
  HostnameFormatDefault: '%stackname%-cephstorage-leaf1-%index%'

    3. Edit the network parameter in the new leaf files so that they align with the respective Leaf network parameters.

      Example

      For example, the parameters in the Leaf 1 Compute file have the following values: 

      - name: ComputeLeaf1
  networks:
    InternalApi:
      subnet: internal_api_leaf1
    Tenant:
      subnet: tenant_leaf1
    Storage:
      subnet: storage_leaf1

      Example

      The Leaf 1 Ceph Storage parameters have the following values: 

      - name: CephStorageLeaf1
  networks:
    Storage:
      subnet: storage_leaf1
    StorageMgmt:
      subnet: storage_mgmt_leaf1

    4. When your role configuration is complete, run the following command to generate the full roles data file. Include all of the leafs in your network and the new leaf that you are adding.

      Example

      In this example, leaf3 is added to leaf0, leaf1, and leaf2: 

      $ openstack overcloud roles generate --roles-path ~/roles -o roles_data_spine_leaf.yaml Controller Controller1 Controller2 Compute Compute1 Compute2 Compute3 CephStorage CephStorage1 CephStorage2 CephStorage3

      This creates a full roles_data_spine_leaf.yaml file that includes all of the custom roles for each respective leaf network.

  5. Create a custom NIC configuration for the leaf that you are adding.

    1. Copy a leaf Compute and a leaf Ceph Storage NIC configuration file for the new leaf that you are adding.

      Example

      In this example, computeleaf1.yaml and ceph-storageleaf1.yaml are copied for the new leaf, computeleaf3.yaml and ceph-storageleaf3.yaml, respectively: 

      $ cp ~/templates/spine-leaf-nics/computeleaf1.yaml ~/templates/spine-leaf-nics/computeleaf3.yaml
$ cp ~/templates/spine-leaf-nics/ceph-storageleaf1.yaml ~/templates/spine-leaf-nics/ceph-storageleaf3.yaml

  6. Open your custom environment file that contains the roles and the custom NIC template mappings for each role, for example, spine-leaf-nic-roles-map.yaml. Insert an entry for each role for the new leaf that you are adding. 

    parameter_defaults:
  %%ROLE%%NetworkConfigTemplate: <path_to_ansible_jinja2_nic_config_file>

    Example

    In this example, the entries ComputeLeaf3NetworkConfigTemplate and CephStorage3NetworkConfigTemplate have been added: 

    parameter_defaults:
  Controller0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  Controller1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  Controller2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  ComputeLeaf3NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage0NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage1NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage2NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'
  CephStorage3NetworkConfigTemplate: '/home/stack/templates/spine-leaf-nics/single-nic-vlans.j2'

  7. Open your custom network environment file that contains the separate network mappings and sets access to the control plane networks for the overcloud, for example, spine-leaf-ctlplane.yaml and update the control plane parameters.

    Under the parameter_defaults section, add the control plane subnet mapping for the new leaf network. Also, include the external network mapping for the new leaf network.

    • For flat network mappings, list the new leaf (leaf3) in the NeutronFlatNetworks parameter and set the NeutronBridgeMappings parameter for the new leaf: 

      parameter_defaults:
  NeutronFlatNetworks: leaf0,leaf1,leaf2,leaf3
  Controller0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  Compute0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  Compute1Parameters:
    NeutronBridgeMappings: "leaf1:br-ex"
  Compute2Parameters:
    NeutronBridgeMappings: "leaf2:br-ex"
  Compute3Parameters:
    NeutronBridgeMappings: "leaf3:br-ex"

    • For VLAN network mappings, additionally set the NeutronNetworkVLANRanges to map VLANs for the new leaf (leaf3) network: 

        NeutronNetworkType: 'geneve,vlan'
  NeutronNetworkVLANRanges: 'leaf0:1:1000,leaf1:1:1000,leaf2:1:1000,leaf3:1:1000'

      Example

      In this example, flat network mappings are used, and the new leaf (leaf3) entries are added: 

      parameter_defaults:
  NeutronFlatNetworks: leaf0,leaf1,leaf2,leaf3
  Controller0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  ControllerControlPlaneSubnet: leaf0
  Controller1Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  Controller1ControlPlaneSubnet: leaf0
  Controller2Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  Controller2ControlPlaneSubnet: leaf0
  Compute0Parameters:
    NeutronBridgeMappings: "leaf0:br-ex"
  Compute0ControlPlaneSubnet: leaf0
  Compute1Parameters:
    NeutronBridgeMappings: "leaf1:br-ex"
  Compute1ControlPlaneSubnet: leaf1
  Compute2Parameters:
    NeutronBridgeMappings: "leaf2:br-ex"
  Compute2ControlPlaneSubnet: leaf2
  Compute3Parameters:
    NeutronBridgeMappings: "leaf3:br-ex"
  Compute3ControlPlaneSubnet: leaf3
  8. Redeploy your spine-leaf enabled overcloud, by following the steps in Deploying a spine-leaf enabled overcloud.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat, Inc.