Chapter 6. Deploying SR-IOV technologies

Procedure

1. Generate the ComputeSriov role:

   openstack overcloud roles generate -o roles_data.yaml Controller ComputeSriov

   Copy to Clipboard Toggle word wrap

2. Configure the physical_network parameter to match your environment.

   • For VLAN, set the physical_network parameter to the name of the network you create in neutron after deployment. This value should also be in NeutronBridgeMappings.
   • For VXLAN, set the physical_network parameter to the string value null.

   • Ensure the OvsHwOffload parameter under role specific parameters has a value of true.

     Example:

     parameter_defaults:
       ComputeSriovParameters:
         IsolCpusList: 2-9,21-29,11-19,31-39
         KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=128 intel_iommu=on iommu=pt"
         OvsHwOffload: true
         TunedProfileName: "cpu-partitioning"
         NeutronBridgeMappings:
           - tenant:br-tenant
         NeutronPhysicalDevMappings:
           - tenant:p7p1
           - tenant:p7p2
         NovaPCIPassthrough:
           - devname: "p7p1"
             physical_network: "null"
           - devname: "p7p2"
             physical_network: "null"
         NovaReservedHostMemory: 4096
         NovaVcpuPinSet: 1-9,21-29,11-19,31-39

     Copy to Clipboard Toggle word wrap

3. Ensure that the list of default filters includes the value NUMATopologyFilter:

     NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']

   Copy to Clipboard Toggle word wrap

4. Configure one or more network interfaces intended for hardware offload in the compute-sriov.yaml configuration file:

   Note

   Do not use the NeutronSriovNumVFs parameter when configuring Open vSwitch hardware offload. The number of virtual functions will be specified using the numvfs parameter in a network configuration file used by os-net-config. Red Hat does not support modifying the numvfs setting during update or redeployment.

     - type: ovs_bridge
       name: br-tenant
       mtu: 9000
       members:
       - type: sriov_pf
         name: p7p1
         numvfs: 5
         mtu: 9000
         primary: true
         promisc: true
         use_dhcp: false
         link_mode: switchdev

   Copy to Clipboard Toggle word wrap
   Note

   Do not configure Mellanox network interfaces as a nic-config interface type ovs-vlan because this prevents tunnel endpoints such as VXLAN from passing traffic due to driver limitations.

5. Include the following files during the deployment of the overcloud:

   • ovs-hw-offload.yaml

   • host-config-and-reboot.yaml

     TEMPLATES_HOME=”/usr/share/openstack-tripleo-heat-templates”
     CUSTOM_TEMPLATES=”/home/stack/templates”
     
     openstack overcloud deploy --templates \
       -r ${CUSTOME_TEMPLATES}/roles_data.yaml \
       -e ${TEMPLATES_HOME}/environments/ovs-hw-offload.yaml \
       -e ${TEMPLATES_HOME}/environments/host-config-and-reboot.yaml \
       -e ${CUSTOME_TEMPLATES}/network-environment.yaml

     Copy to Clipboard Toggle word wrap

1. Confirm that a pci device has its mode configured as switchdev:

   # devlink dev eswitch show pci/0000:03:00.0
   pci/0000:03:00.0: mode switchdev inline-mode none encap enable

   Copy to Clipboard Toggle word wrap

2. Confirm offload is enabled in OVS:

   # ovs-vsctl get Open_vSwitch . other_config:hw-offload
   "true"

   Copy to Clipboard Toggle word wrap

3. Confirm hardware offload is enabled on the NIC:

   # ethtool -k $NIC | grep tc-offload
   hw-tc-offload: on

   Copy to Clipboard Toggle word wrap

1. Ensure that the AggregateInstanceExtraSpecsFilter value is included in the scheduler_default_filters parameter in the nova.conf configuration file. This configuration can be set through the heat parameter NovaSchedulerDefaultFilters under role-specific parameters before deployment.

     ComputeOvsDpdkSriovParameters:
       NovaSchedulerDefaultFilters: ['AggregateInstanceExtraSpecsFilter', 'RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']

   Copy to Clipboard Toggle word wrap
   Note

   This parameter can be added to heat templates and the original deployment script re-run to add this to the configuration of an exiting cluster.

2. Create an aggregate group for single root I/O virtualization (SR-IOV), and add relevant hosts. Define metadata, for example, sriov=true, that matches defined flavor metadata.

   # openstack aggregate create sriov_group
   # openstack aggregate add host sriov_group compute-sriov-0.localdomain
   # openstack aggregate set sriov_group sriov=true

   Copy to Clipboard Toggle word wrap

3. Create a flavor.

   # openstack flavor create <flavor> --ram <MB> --disk <GB> --vcpus <#>

   Copy to Clipboard Toggle word wrap

4. Set additional flavor properties. Note that the defined metadata, sriov=true, matches the defined metadata on the SR-IOV aggregate.

   openstack flavor set --property sriov=true --property hw:cpu_policy=dedicated --property hw:mem_page_size=1GB <flavor>

   Copy to Clipboard Toggle word wrap