Chapter 2. Configuring routed spine-leaf in the undercloud

Procedure

1. Log in to the undercloud as the stack user.

2. If you do not already have an undercloud.conf file, copy the sample template file:

   [stack@director ~]$ cp /usr/share/python-tripleoclient/undercloud.conf.sample ~/undercloud.conf

   Copy to Clipboard Toggle word wrap
3. Edit the undercloud.conf file.

4. Set the following values in the [DEFAULT] section:

   1. Set local_ip to the undercloud IP on leaf0:

      local_ip = 192.168.10.1/24

      Copy to Clipboard Toggle word wrap

   2. Set undercloud_public_host to the externally facing IP address of the undercloud:

      undercloud_public_host = 10.1.1.1

      Copy to Clipboard Toggle word wrap

   3. Set undercloud_admin_host to the administration IP address of the undercloud. This IP address is usually on leaf0:

      undercloud_admin_host = 192.168.10.2

      Copy to Clipboard Toggle word wrap

   4. Set local_interface to the interface to bridge for the local network:

      local_interface = eth1

      Copy to Clipboard Toggle word wrap

   5. Set enable_routed_networks to true:

      enable_routed_networks = true

      Copy to Clipboard Toggle word wrap

   6. Define your list of subnets using the subnets parameter. Define one subnet for each L2 segment in the routed spine and leaf:

      subnets = leaf0,leaf1,leaf2

      Copy to Clipboard Toggle word wrap

   7. Specify the subnet associated with the physical L2 segment local to the undercloud using the local_subnet parameter:

      local_subnet = leaf0

      Copy to Clipboard Toggle word wrap

   8. Set the value of undercloud_nameservers.

      undercloud_nameservers = 10.11.5.19,10.11.5.20

      Copy to Clipboard Toggle word wrap
      Tip

      You can find the current IP addresses of the DNS servers that are used for the undercloud nameserver by looking in /etc/resolv.conf.

5. Create a new section for each subnet that you define in the subnets parameter:

   [leaf0]
   cidr = 192.168.10.0/24
   dhcp_start = 192.168.10.10
   dhcp_end = 192.168.10.90
   inspection_iprange = 192.168.10.100,192.168.10.190
   gateway = 192.168.10.1
   masquerade = False
   
   [leaf1]
   cidr = 192.168.11.0/24
   dhcp_start = 192.168.11.10
   dhcp_end = 192.168.11.90
   inspection_iprange = 192.168.11.100,192.168.11.190
   gateway = 192.168.11.1
   masquerade = False
   
   [leaf2]
   cidr = 192.168.12.0/24
   dhcp_start = 192.168.12.10
   dhcp_end = 192.168.12.90
   inspection_iprange = 192.168.12.100,192.168.12.190
   gateway = 192.168.12.1
   masquerade = False

   Copy to Clipboard Toggle word wrap
6. Save the undercloud.conf file.

7. Run the undercloud installation command:

   [stack@director ~]$ openstack undercloud install

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the stackrc file:

   [stack@director ~]$ source ~/stackrc

   Copy to Clipboard Toggle word wrap

2. Create flavors for each custom role:

   $ ROLES="control compute_leaf0 compute_leaf1 compute_leaf2 ceph-storage_leaf0 ceph-storage_leaf1 ceph-storage_leaf2"
   $ for ROLE in $ROLES; do openstack flavor create --id auto --ram <ram_size_mb> --disk <disk_size_gb> --vcpus <no_vcpus> $ROLE ; done
   $ for ROLE in $ROLES; do openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property resources:DISK_GB='0' --property resources:MEMORY_MB='0' --property resources:VCPU='0' $ROLE ; done

   Copy to Clipboard Toggle word wrap
   • Replace <ram_size_mb> with the RAM of the bare metal node, in MB.
   • Replace <disk_size_gb> with the size of the disk on the bare metal node, in GB.
   • Replace <no_vcpus> with the number of CPUs on the bare metal node.

3. Retrieve a list of your nodes to identify their UUIDs:

   (undercloud)$ openstack baremetal node list

   Copy to Clipboard Toggle word wrap

4. Tag each bare metal node to its leaf network and role by using a custom resource class:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.LEAF-ROLE <node>

   Copy to Clipboard Toggle word wrap

   Replace <node> with the ID of the bare metal node.

   For example, enter the following command to tag a node with UUID 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13 to the Compute role on Leaf2:

   (undercloud)$ openstack baremetal node set \
    --resource-class baremetal.COMPUTE-LEAF2 58c3d07e-24f2-48a7-bbb6-6843f0e8ee13

   Copy to Clipboard Toggle word wrap

5. Associate each leaf network role flavor with the custom resource class:

   (undercloud)$ openstack flavor set \
    --property resources:CUSTOM_BAREMETAL_LEAF_ROLE=1 \
   <custom_role>

   Copy to Clipboard Toggle word wrap

   To determine the name of a custom resource class that corresponds to a resource class of a Bare Metal Provisioning service node, convert the resource class to uppercase, replace each punctuation mark with an underscore, and prefix with CUSTOM_.

   Note

   A flavor can request only one instance of a bare metal resource class.

6. In the node-info.yaml file, specify the flavor that you want to use for each custom leaf role, and the number of nodes to allocate for each custom leaf role. For example, the following configuration specifies the flavor to use, and the number of nodes to allocate for the custom leaf roles compute_leaf0, compute_leaf1, compute_leaf2, ceph-storage_leaf0, ceph-storage_leaf1, and ceph-storage_leaf2:

   parameter_defaults:
     OvercloudControllerFlavor: control
     OvercloudComputeLeaf0Flavor: compute_leaf0
     OvercloudComputeLeaf1Flavor: compute_leaf1
     OvercloudComputeLeaf2Flavor: compute_leaf2
     OvercloudCephStorageLeaf0Flavor: ceph-storage_leaf0
     OvercloudCephStorageLeaf1Flavor: ceph-storage_leaf1
     OvercloudCephStorageLeaf2Flavor: ceph-storage_leaf2
     ControllerLeaf0Count: 3
     ComputeLeaf0Count: 3
     ComputeLeaf1Count: 3
     ComputeLeaf2Count: 3
     CephStorageLeaf0Count: 3
     CephStorageLeaf1Count: 3
     CephStorageLeaf2Count: 3

   Copy to Clipboard Toggle word wrap

Procedure

1. Source the stackrc file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap

2. Check the bare metal nodes:

   $ openstack baremetal node list

   Copy to Clipboard Toggle word wrap

3. Ensure that the bare metal nodes are either in enroll or manageable state. If the bare metal node is not in one of these states, the command that sets the physical_network property on the baremetal port fails. To set all nodes to manageable state, run the following command:

   $ for node in $(openstack baremetal node list -f value -c Name); do openstack baremetal node manage $node --wait; done

   Copy to Clipboard Toggle word wrap

4. Check which baremetal ports are associated with which baremetal node:

   $ openstack baremetal port list --node <node-uuid>

   Copy to Clipboard Toggle word wrap

5. Set the physical-network parameter for the ports. In the example below, three subnets are defined in the configuration: leaf0, leaf1, and leaf2. The local_subnet is leaf0. Because the physical network for the local_subnet is always ctlplane, the baremetal port connected to leaf0 uses ctlplane. The remaining ports use the other leaf names:

   $ openstack baremetal port set --physical-network ctlplane <port-uuid>
   $ openstack baremetal port set --physical-network leaf1 <port-uuid>
   $ openstack baremetal port set --physical-network leaf2 <port-uuid>

   Copy to Clipboard Toggle word wrap

6. Introspect the nodes before you deploy the overcloud. Include the --all-manageable and --provide options to set the nodes as available for deployment:

   $ openstack overcloud node introspect --all-manageable --provide

   Copy to Clipboard Toggle word wrap

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the undercloud credentials file:

   $ source ~/stackrc

   Copy to Clipboard Toggle word wrap

3. In the /home/stack/undercloud.conf file, do the following:

   1. Locate the subnets parameter, and add a new subnet for the leaf that you are adding.

      A subnet represents an L2 segment in the routed spine and leaf:

      Example

      In this example, a new subnet (leaf3) is added for the new leaf (leaf3):

      subnets = leaf0,leaf1,leaf2,leaf3

      Copy to Clipboard Toggle word wrap

   2. Create a section for the subnet that you added.

      Example

      In this example, the section [leaf3] is added for the new subnet (leaf3):

      [leaf0]
      cidr = 192.168.10.0/24
      dhcp_start = 192.168.10.10
      dhcp_end = 192.168.10.90
      inspection_iprange = 192.168.10.100,192.168.10.190
      gateway = 192.168.10.1
      masquerade = False
      
      [leaf1]
      cidr = 192.168.11.0/24
      dhcp_start = 192.168.11.10
      dhcp_end = 192.168.11.90
      inspection_iprange = 192.168.11.100,192.168.11.190
      gateway = 192.168.11.1
      masquerade = False
      
      [leaf2]
      cidr = 192.168.12.0/24
      dhcp_start = 192.168.12.10
      dhcp_end = 192.168.12.90
      inspection_iprange = 192.168.12.100,192.168.12.190
      gateway = 192.168.12.1
      masquerade = False
      
      [leaf3]
      cidr = 192.168.13.0/24
      dhcp_start = 192.168.13.10
      dhcp_end = 192.168.13.90
      inspection_iprange = 192.168.13.100,192.168.13.190
      gateway = 192.168.13.1
      masquerade = False

      Copy to Clipboard Toggle word wrap
4. Save the undercloud.conf file.

5. Reinstall your undercloud:

   $ openstack undercloud install

   Copy to Clipboard Toggle word wrap