Network Functions Virtualization Planning and Configuration Guide

Procedure

1. Disable the default repositories.

   subscription-manager repos --disable=*

   Copy to Clipboard Toggle word wrap

2. Enable required repositories for Red Hat OpenStack Platform with network functions virtualization (NFV).

   sudo subscription-manager repos \
   --enable=rhel-7-server-rpms \
   --enable=rhel-7-server-extras-rpms \
   --enable=rhel-7-server-rh-common-rpms \
   --enable=rhel-ha-for-rhel-7-server-rpms \
   --enable=rhel-7-server-openstack-13-rpms \
   --enable=rhel-7-server-nfv-rpms

   Copy to Clipboard Toggle word wrap

1. Generate the built-in ComputeSriov to define nodes in the OpenStack cluster that will run NeutronSriovAgent, NeutronSriovHostConfig and default compute services.

   # openstack overcloud roles generate \
   -o /home/stack/templates/roles_data.yaml \
   Controller ComputeSriov

   Copy to Clipboard Toggle word wrap

2. Include the neutron-sriov.yaml and roles_data.yaml files when generating overcloud_images.yaml so that SR-IOV containers are prepared.

   SERVICES=\
   /usr/share/openstack-tripleo-heat-templates/environments/services
   
   openstack overcloud container image prepare \
   --namespace=registry.redhat.io/rhosp13 \
   --push-destination=192.168.24.1:8787 \
   --prefix=openstack- \
   --tag-from-label {version}-{release} \
   -e ${SERVICES}/neutron-sriov.yaml \
   --roles-file /home/stack/templates/roles_data.yaml \
   --output-env-file=/home/stack/templates/overcloud_images.yaml \
   --output-images-file=/home/stack/local_registry_images.yaml

   Copy to Clipboard Toggle word wrap
   Note

   The push-destination IP address is the address that you previously set with the local_ip parameter in the undercloud.conf configuration file.

   For more information on container image preparation, see Director Installation and Usage.

3. To apply the KernelAgs and TunedProfile parameters, include the host-config-and-reboot.yaml file from /usr/share/openstack-tripleo-heat-templates/environments to your deployment script.

   openstack overcloud deploy --templates \
   … \
   -e /usr/share/openstack-tripleo-heat-templates/environments/host-config-and-reboot.yaml \
   ...

   Copy to Clipboard Toggle word wrap

4. Configure the parameters for the SR-IOV nodes under parameter_defaults in accordance with the needs of your cluster, and the configuration of your hardware. These settings are typically added to the network-environment.yaml file.

     NeutronNetworkType: 'vlan'
     NeutronNetworkVLANRanges:
       - tenant:22:22
       - tenant:25:25
     NeutronTunnelTypes: ''

   Copy to Clipboard Toggle word wrap

5. In the same file, configure role specific parameters for SR-IOV compute nodes.

   Note

   The NeutronSriovNumVFs parameter will soon be deprecated in favor of the numvfs attribute in the network configuration templates. Red Hat does not support modification of the NeutronSriovNumVFs parameter, nor the numvfs parameter, after deployment. Changing either parameter within a running environment is known to cause a permanent outage for all running instances which have an SR-IOV port on that PF. Unless you hard reboot these instances, the SR-IOV PCI device will not be visible to the instance.

     ComputeSriovParameters:
       IsolCpusList: "1-19,21-39"
       KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-19,21-39"
       TunedProfileName: "cpu-partitioning"
       NeutronBridgeMappings:
         - tenant:br-link0
       NeutronPhysicalDevMappings:
         - tenant:p7p1
         - tenant:p7p2
       NeutronSriovNumVFs:
         - p7p1:5
         - p7p2:5
       NovaPCIPassthrough:
         - vendor_id: "8086"
           product_id: "1528"
           address: "0000:06:00.0"
           physical_network: "tenant"
         - vendor_id: "8086"
           product_id: "1528"
           address: "0000:06:00.1"
           physical_network: "tenant"
       NovaVcpuPinSet: '1-19,21-39'
       NovaReservedHostMemory: 4096

   Copy to Clipboard Toggle word wrap
   Note

   Do not use the devname parameter when configuring PCI passthrough, as the device name of a NIC can change. Instead, use vendor_id and product_id because they are more stable, or use the address of the NIC. For more information about how to configure NovaPCIPassthrough, see Guidelines for configuring NovaPCIPassthrough.

6. Configure the SR-IOV enabled interfaces in the compute.yaml network configuration template. Ensure the interfaces are configured as standalone NICs for the purposes of creating SR-IOV virtual functions (VFs):

                - type: interface
                   name: p7p3
                   mtu: 9000
                   use_dhcp: false
                   defroute: false
                   nm_controlled: true
                   hotplug: true
   
                 - type: interface
                   name: p7p4
                   mtu: 9000
                   use_dhcp: false
                   defroute: false
                   nm_controlled: true
                   hotplug: true

   Copy to Clipboard Toggle word wrap

7. Ensure that the list of default filters includes the value AggregateInstanceExtraSpecsFilter.

   NovaSchedulerDefaultFilters: ['AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','AggregateInstanceExtraSpecsFilter']

   Copy to Clipboard Toggle word wrap
8. Deploy the overcloud.

Procedure

1. Generate the ComputeSriov role:

   openstack overcloud roles generate -o roles_data.yaml Controller ComputeSriov

   Copy to Clipboard Toggle word wrap
2. Add the OvsHwOffload parameter under role-specific parameters with a value of true.
3. To configure neutron to use the iptables/hybrid firewall driver implementation, include the line: NeutronOVSFirewallDriver: iptables_hybrid. For more information about NeutronOVSFirewallDriver, see Using the Open vSwitch Firewall in the Advanced Overcloud Customization Guide.

4. Configure the physical_network parameter to match your environment.

   • For VLAN, set the physical_network parameter to the name of the network you create in neutron after deployment. This value should also be in NeutronBridgeMappings.

   • For VXLAN, set the physical_network parameter to null.

     Example:

     parameter_defaults:
       NeutronOVSFirewallDriver: iptables_hybrid
       ComputeSriovParameters:
         IsolCpusList: 2-9,21-29,11-19,31-39
         KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=128 intel_iommu=on iommu=pt"
         OvsHwOffload: true
         TunedProfileName: "cpu-partitioning"
         NeutronBridgeMappings:
           - tenant:br-tenant
         NovaPCIPassthrough:
           - vendor_id: <vendor-id>
             product_id: <product-id>
             address: <address>
             physical_network: "tenant"
           - vendor_id: <vendor-id>
             product_id: <product-id>
             address: <address>
             physical_network: "null"
         NovaReservedHostMemory: 4096
         NovaComputeCpuDedicatedSet: 1-9,21-29,11-19,31-39

     Copy to Clipboard Toggle word wrap
   • Replace <vendor-id> with the vendor ID of the physical NIC.
   • Replace <product-id> with the product ID of the NIC VF.

   • Replace <address> with the address of the physical NIC.

     For more information about how to configure NovaPCIPassthrough, see Guidelines for configuring NovaPCIPassthrough.

5. Ensure that the list of default filters includes NUMATopologyFilter:

     NovaSchedulerDefaultFilters: [\'RetryFilter',\'AvailabilityZoneFilter',\'ComputeFilter',\'ComputeCapabilitiesFilter',\'ImagePropertiesFilter',\'ServerGroupAntiAffinityFilter',\'ServerGroupAffinityFilter',\'PciPassthroughFilter',\'NUMATopologyFilter']

   Copy to Clipboard Toggle word wrap

6. Configure one or more network interfaces intended for hardware offload in the compute-sriov.yaml configuration file:

     - type: ovs_bridge
       name: br-tenant
       mtu: 9000
       members:
       - type: sriov_pf
         name: p7p1
         numvfs: 5
         mtu: 9000
         primary: true
         promisc: true
         use_dhcp: false
         link_mode: switchdev

   Copy to Clipboard Toggle word wrap
   Note
   • Do not use the NeutronSriovNumVFs parameter when configuring Open vSwitch hardware offload. The number of virtual functions is specified using the numvfs parameter in a network configuration file used by os-net-config. Red Hat does not support modifying the numvfs setting during update or redeployment.
   • Do not configure Mellanox network interfaces as a nic-config interface type ovs-vlan because this prevents tunnel endpoints such as VXLAN from passing traffic due to driver limitations.

7. Include the ovs-hw-offload.yaml file in the overcloud deploy command:

   TEMPLATES_HOME=”/usr/share/openstack-tripleo-heat-templates”
   CUSTOM_TEMPLATES=”/home/stack/templates”
   
   openstack overcloud deploy --templates \
     -r ${CUSTOM_TEMPLATES}/roles_data.yaml \
     -e ${TEMPLATES_HOME}/environments/ovs-hw-offload.yaml \
     -e ${CUSTOM_TEMPLATES}/network-environment.yaml \
     -e ${CUSTOM_TEMPLATES}/neutron-ovs.yaml

   Copy to Clipboard Toggle word wrap

1. Create a flavor.

   # openstack flavor create <flavor> --ram <MB> --disk <GB> --vcpus <#>

   Copy to Clipboard Toggle word wrap

2. Create the network.

   # openstack network create net1 --provider-physical-network tenant --provider-network-type vlan --provider-segment <VLAN-ID>
   # openstack subnet create subnet1 --network net1 --subnet-range 192.0.2.0/24 --dhcp

   Copy to Clipboard Toggle word wrap

3. Create the port.

   • Use vnic-type direct to create an SR-IOV virtual function (VF) port.

     # openstack port create --network net1 --vnic-type direct sriov_port

     Copy to Clipboard Toggle word wrap

   • Use the following to create a virtual function with hardware offload.

     # openstack port create --network net1 --vnic-type direct --binding-profile '{"capabilities": ["switchdev"]} sriov_hwoffload_port

     Copy to Clipboard Toggle word wrap

   • Use vnic-type direct-physical to create an SR-IOV PF port.

     # openstack port create --network net1 --vnic-type direct-physical sriov_port

     Copy to Clipboard Toggle word wrap

4. Deploy an instance

   # openstack server create --flavor <flavor> --image <image> --nic port-id=<id> <instance name>

   Copy to Clipboard Toggle word wrap

1. You can configure the AggregateInstanceExtraSpecsFilter value, and other necessary filters, through the heat parameter NovaSchedulerDefaultFilters under parameter_defaults in the nova.conf configuration file before deployment.

     parameter_defaults:
       NovaSchedulerDefaultFilters: ['AggregateInstanceExtraSpecsFilter', 'RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']

   Copy to Clipboard Toggle word wrap
   Note

   To add the AggregateInstanceExtraSpecsFilter configuration to an exiting cluster, you can add this parameter to the heat templates, and run the original deployment script again.

2. Create an aggregate group for single root I/O virtualization (SR-IOV), and add relevant hosts. Define metadata, for example, sriov=true, that matches defined flavor metadata.

   # openstack aggregate create sriov_group
   # openstack aggregate add host sriov_group compute-sriov-0.localdomain
   # openstack aggregate set --property sriov=true sriov_group

   Copy to Clipboard Toggle word wrap

3. Create a flavor.

   # openstack flavor create <flavor> --ram <MB> --disk <GB> --vcpus <#>

   Copy to Clipboard Toggle word wrap

4. Set additional flavor properties. Note that the defined metadata, sriov=true, matches the defined metadata on the SR-IOV aggregate.

   openstack flavor set --property sriov=true --property hw:cpu_policy=dedicated --property hw:mem_page_size=1GB <flavor>

   Copy to Clipboard Toggle word wrap

1. Copy the /usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-params.yaml file to a local directory and set the input parameters to suit your environment.

     workflow_parameters:
       tripleo.derive_params.v1.derive_parameters:
         EmptyEmpty# DPDK Parameters Empty#
         # Specifices the minimum number of CPU physical cores to be allocated for DPDK
         # PMD threads. The actual allocation will be based on network config, if
         # the a DPDK port is associated with a numa node, then this configuration
         # will be used, else 1.
         num_phy_cores_per_numa_node_for_pmd: 1
         # Amount of memory to be configured as huge pages in percentage. Ouf the
         # total available memory (excluding the NovaReservedHostMemory), the
         # specified percentage of the remaining is configured as huge pages.
         huge_page_allocation_percentage: 50

   Copy to Clipboard Toggle word wrap

2. Run the openstack overcloud deploy command and include the following:

   • The update-plan-only option
   • The role file and all environment files specific to your environment

   • The plan-environment-derived-parms.yaml file with the --plan-environment-file optional argument

     $ openstack overcloud deploy --templates --update-plan-only \
     -r /home/stack/roles_data.yaml \
     -e /home/stack/<environment-file> \
     ... #repeat as necessary ...
     -p /home/stack/plan-environment-derived-params.yaml

     Copy to Clipboard Toggle word wrap

1. Copy the derived parameters from the plan-environment.yaml to the network-environment.yaml file.

     # DPDK compute node.
     ComputeOvsDpdkParameters:
       KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-7,17-23,9-15,25-31
       TunedProfileName: "cpu-partitioning"
       IsolCpusList: "1-7,17-23,9-15,25-31"
       NovaVcpuPinSet: ['2-7,18-23,10-15,26-31']
       NovaReservedHostMemory: 4096
       OvsDpdkSocketMemory: "1024,1024"
       OvsDpdkMemoryChannels: "4"
       OvsDpdkCoreList: "0,16,8,24"
       OvsPmdCoreList: "1,17,9,25"

   Copy to Clipboard Toggle word wrap
   Note

   These parameters apply exclusively to the role ComputeOvsDpdk, and will not apply to other roles, including Compute or ComputeSriov that may exist on the same cluster. You can apply these parameters globally, but any global parameters are overwritten by role-specific parameters.

2. Deploy the overcloud using the role file and all environment files specific to your environment. See Deploying the Overcloud for details.

1. Set the NeutronGlobalPhysnetMtu parameter in the network-environment.yaml file.

   parameter_defaults:
     # MTU global configuration
     NeutronGlobalPhysnetMtu: 9000

   Copy to Clipboard Toggle word wrap
   Note

   Ensure that the NeutronDpdkSocketMemory value in the network-environment.yaml file is large enough to support jumbo frames. See Section 7.4.2, “Memory parameters” for details.

2. Set the MTU value on the bridge to the Compute node in the controller.yaml file.

     -
       type: ovs_bridge
       name: br-link0
       use_dhcp: false
       members:
         -
           type: interface
           name: nic3
           mtu: 9000

   Copy to Clipboard Toggle word wrap

3. Set the MTU values for an OVS-DPDK bond in the compute.yaml file:

   - type: ovs_user_bridge
     name: br-link0
     use_dhcp: false
     members:
       - type: ovs_dpdk_bond
         name: dpdkbond0
         mtu: 9000
         rx_queue: 2
         members:
           - type: ovs_dpdk_port
             name: dpdk0
             mtu: 9000
             members:
               - type: interface
                 name: nic4
           - type: ovs_dpdk_port
             name: dpdk1
             mtu: 9000
             members:
               - type: interface
                 name: nic5

   Copy to Clipboard Toggle word wrap

1. Ensure parameters for your DPDK compute role are populated in network-environment.yaml. These can be copied from derived OVS-DPDK parameters if needed:

    # DPDK compute node.
     ComputeOvsDpdkParameters:
       KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-7,17-23,9-15,25-31
       TunedProfileName: "cpu-partitioning"
       IsolCpusList: "1-7,17-23,9-15,25-31"
       NovaVcpuPinSet: ['2-7,18-23,10-15,26-31']
       NovaReservedHostMemory: 4096
       OvsDpdkSocketMemory: "1024,1024"
       OvsDpdkMemoryChannels: "4"
       OvsDpdkCoreList: "0,16,8,24"
       OvsPmdCoreList: "1,17,9,25"

   Copy to Clipboard Toggle word wrap

2. Deploy the overcloud using the openstack overcloud deploy command.

   • Include the role file and all environment files specific to your environment.

   • Apply the KernelArgs and TunedProfile parameters by including the host-config-and-reboot.yaml file from /usr/share/openstack-tripleo-heat-templates/environments to your deployment script:

     TEMPLATES_HOME=”/usr/share/openstack-tripleo-heat-templates”
     CUSTOM_TEMPLATES=”/home/stack/templates”
     
     openstack overcloud deploy --templates \
     -r ${CUSTOM_TEMPLATES}/roles_data.yaml \
     -e ${TEMPLATES_HOME}/environments/host-config-and-reboot.yaml \
     -e ${CUSTOM_TEMPLATES}/network-environment.yaml \
     -e ${CUSTOM_TEMPLATES}/controller.yaml
     -e ${CUSTOM_TEMPLATES}/computeovsdpdk.yaml \
     ...

     Copy to Clipboard Toggle word wrap

1. Create an aggregate group and add relevant hosts for OVS-DPDK. Define metadata, for example dpdk=true, that matches defined flavor metadata.

    # openstack aggregate create dpdk_group
    # openstack aggregate add host dpdk_group [compute-host]
    # openstack aggregate set --property dpdk=true dpdk_group

   Copy to Clipboard Toggle word wrap
   Note

   You should use host aggregates to separate CPU pinned instances from unpinned instances. Instances that do not use CPU pinning do not respect the resourcing requirements of instances that use CPU pinning.

2. Create a flavor.

   # openstack flavor create <flavor> --ram <MB> --disk <GB> --vcpus <#>

   Copy to Clipboard Toggle word wrap

3. Set additional flavor properties. Note that the defined metadata, dpdk=true, matches the defined metadata in the DPDK aggregate.

   # openstack flavor set <flavor> --property dpdk=true --property hw:cpu_policy=dedicated --property hw:mem_page_size=1GB --property hw:emulator_threads_policy=isolate

   Copy to Clipboard Toggle word wrap

   For details on the emulator threads policy for performance improvements, see: Configure Emulator Threads to run on a Dedicated Physical CPU.

4. Create the network.

   # openstack network create net1 --provider-physical-network tenant --provider-network-type vlan --provider-segment <VLAN-ID>
   # openstack subnet create subnet1 --network net1 --subnet-range 192.0.2.0/24 --dhcp

   Copy to Clipboard Toggle word wrap

5. Optional: If you use multiqueue with OVS-DPDK, set the hw_vif_multiqueue_enabled property on the image that you want to use to create a instance:

   # openstack image set --property hw_vif_multiqueue_enabled=true <image>

   Copy to Clipboard Toggle word wrap

6. Deploy an instance.

   # openstack server create --flavor <flavor> --image <glance image> --nic net-id=<network ID> <server_name>

   Copy to Clipboard Toggle word wrap

1. Review the bridge configuration and confirm that the bridge was created with the datapath_type=netdev.

   # ovs-vsctl list bridge br0
   _uuid               : bdce0825-e263-4d15-b256-f01222df96f3
   auto_attach         : []
   controller          : []
   datapath_id         : "00002608cebd154d"
   datapath_type       : netdev
   datapath_version    : "<built-in>"
   external_ids        : {}
   fail_mode           : []
   flood_vlans         : []
   flow_tables         : {}
   ipfix               : []
   mcast_snooping_enable: false
   mirrors             : []
   name                : "br0"
   netflow             : []
   other_config        : {}
   ports               : [52725b91-de7f-41e7-bb49-3b7e50354138]
   protocols           : []
   rstp_enable         : false
   rstp_status         : {}
   sflow               : []
   status              : {}
   stp_enable          : false

   Copy to Clipboard Toggle word wrap

2. Confirm that the docker container neutron_ovs_agent is configured to start automatically.

   # docker inspect neutron_ovs_agent | grep -A1 RestartPolicy
               "RestartPolicy": {
                   "Name": "always",

   Copy to Clipboard Toggle word wrap

   If the container is having trouble starting, you can view any related messages.

   # less /var/log/containers/neutron/openvswitch-agent.log

   Copy to Clipboard Toggle word wrap

3. Confirm that the PMD CPU mask of the ovs-dpdk are pinned to the CPUs. In case of HT, use sibling CPUs.

   For example, take CPU4:

   # cat /sys/devices/system/cpu/cpu4/topology/thread_siblings_list
   4,20

   Copy to Clipboard Toggle word wrap

   So, using CPU 4 and 20:

   # ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x100010

   Copy to Clipboard Toggle word wrap

   Display their status:

   # tuna -t ovs-vswitchd -CP
   thread  ctxt_switches pid SCHED_ rtpri affinity voluntary nonvoluntary       cmd
   3161	OTHER 	0    	6	765023      	614	ovs-vswitchd
   3219   OTHER 	0    	6     	1        	0   	handler24
   3220   OTHER 	0    	6     	1        	0   	handler21
   3221   OTHER 	0    	6     	1        	0   	handler22
   3222   OTHER 	0    	6     	1        	0   	handler23
   3223   OTHER 	0    	6     	1        	0   	handler25
   3224   OTHER 	0    	6     	1        	0   	handler26
   3225   OTHER 	0    	6     	1        	0   	handler27
   3226   OTHER 	0    	6     	1        	0   	handler28
   3227   OTHER 	0    	6     	2        	0   	handler31
   3228   OTHER 	0    	6     	2        	4   	handler30
   3229   OTHER 	0    	6     	2        	5   	handler32
   3230   OTHER 	0    	6	953538      	431   revalidator29
   3231   OTHER 	0    	6   1424258      	976   revalidator33
   3232   OTHER 	0    	6   1424693      	836   revalidator34
   3233   OTHER 	0    	6	951678      	503   revalidator36
   3234   OTHER 	0    	6   1425128      	498   revalidator35
   *3235   OTHER 	0    	4	151123       	51       	pmd37*
   *3236   OTHER 	0   	20	298967       	48       	pmd38*
   3164   OTHER 	0    	6 	47575        	0  dpdk_watchdog3
   3165   OTHER 	0    	6	237634        	0   vhost_thread1
   3166   OTHER 	0    	6  	3665        	0       	urcu2

   Copy to Clipboard Toggle word wrap

1. Install ceph-ansible on the undercloud.

   $ sudo yum install ceph-ansible -y

   Copy to Clipboard Toggle word wrap

2. Generate the roles_data.yaml file for the ComputeHCI role.

   $ openstack overcloud roles generate -o ~/<templates>/roles_data.yaml Controller \
    ComputeHCIOvsDpdk

   Copy to Clipboard Toggle word wrap
3. Create and configure a new flavor with the openstack flavor create and openstack flavor set commands. For more information about creating a flavor, see Creating a new role in the Advanced Overcloud Customization Guide.

4. Deploy the overcloud with the custom roles_data.yaml file that you generated.

   # time openstack overcloud deploy --templates \
    --timeout 360 \
    -r ~/<templates>/roles_data.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/neutron-ovs-dpdk.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/host-config-and-reboot.yaml \
    -e ~/<templates>/<custom environment file>

   Copy to Clipboard Toggle word wrap

1. Add the custom resources for OVS-DPDK under resource_registry:

     resource_registry:
       # Specify the relative/absolute path to the config files you want to use for override the default.
       OS::TripleO::ComputeOvsDpdkSriov::Net::SoftwareConfig:
         nic-configs/computeovsdpdksriov.yaml
       OS::TripleO::Controller::Net::SoftwareConfig:
         nic-configs/controller.yaml

   Copy to Clipboard Toggle word wrap

2. Under parameter_defaults, set the tunnel type and network type to vxlan.:

   NeutronTunnelTypes: 'vxlan'
   NeutronNetworkType: 'vxlan'

   Copy to Clipboard Toggle word wrap

3. Under parameters_defaults, set the bridge mapping:

   # The OVS logical->physical bridge mappings to use.
   NeutronBridgeMappings:
     - dpdk-mgmt:br-link0

   Copy to Clipboard Toggle word wrap

4. Under parameter_defaults, set the role-specific parameters for the ComputeOvsDpdkSriov role:

     ##########################
     # OVS DPDK configuration #
     # ########################
     ComputeOvsDpdkSriovParameters:
       KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2-19,22-39"
       TunedProfileName: "cpu-partitioning"
       IsolCpusList: "2-19,22-39"
       NovaVcpuPinSet: ['4-19,24-39']
       NovaReservedHostMemory: 2048
       OvsDpdkSocketMemory: "3072,1024"
       OvsDpdkMemoryChannels: "4"
       OvsDpdkCoreList: "0,20,1,21"
       OvsPmdCoreList: "2,22,3,23"
       NovaLibvirtRxQueueSize: 1024
       NovaLibvirtTxQueueSize: 1024

   Copy to Clipboard Toggle word wrap
   Note

   To prevent failures during guest creation, assign at least one CPU with sibling thread on each NUMA node. In the example, the values for the OvsPmdCoreList parameter denote cores 2 and 22 from NUMA 0, and cores 3 and 23 from NUMA 1.

   Note

   Huge pages are consumed by virtual machines, as well as OVS-DPDK using the OvsDpdkSocketMemory parameter. To calculate the number of huge pages available to the virtual machine, subtract the OvsDpdkSocketMemory value from the boot parameter value. You must also add hw:mem_page_size=1GB to the flavor you associate with the DPDK instance.

   Note

   OvsDPDKCoreList and OvsDpdkMemoryChannels are required settings for this procedure and must be set correctly to prevent failures.

5. Configure the role-specific parameters for SR-IOV:

     ##########################
     #  SR-IOV configuration  #
     ##########################
     NeutronMechanismDrivers: ['openvswitch','sriovnicswitch']
     NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']
     NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
     NovaPCIPassthrough:
       - vendor_id: "8086"
         product_id: "1528"
         address: "0000:06:00.0"
         trusted: "true"
         physical_network: "sriov-1"
       - vendor_id: "8086"
         product_id: "1528"
         address: "0000:06:00.1"
         trusted: "true"
         physical_network: "sriov-2"

   Copy to Clipboard Toggle word wrap

1. Create the control plane Linux bond for an isolated network.

     - type: linux_bond
       name: bond_api
       bonding_options: "mode=active-backup"
       use_dhcp: false
       dns_servers:
         get_param: DnsServers
       members:
       - type: interface
         name: nic2
         primary: true

   Copy to Clipboard Toggle word wrap

2. Assign VLANs to this Linux bond.

     - type: vlan
       vlan_id:
         get_param: InternalApiNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: InternalApiIpSubnet
   
     - type: vlan
       vlan_id:
         get_param: StorageNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: StorageIpSubnet
   
     - type: vlan
       vlan_id:
         get_param: StorageMgmtNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: StorageMgmtIpSubnet
   
     - type: vlan
       vlan_id:
         get_param: ExternalNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: ExternalIpSubnet
       routes:
       - default: true
         next_hop:
           get_param: ExternalInterfaceDefaultRoute

   Copy to Clipboard Toggle word wrap

3. Create the OVS bridge for access to neutron-dhcp-agent and neutron-metadata-agent services.

     - type: ovs_bridge
       name: br-link0
       use_dhcp: false
       mtu: 9000
       members:
       - type: interface
         name: nic3
         mtu: 9000
       - type: vlan
         vlan_id:
           get_param: TenantNetworkVlanID
         mtu: 9000
         addresses:
         - ip_netmask:
             get_param: TenantIpSubnet

   Copy to Clipboard Toggle word wrap

1. Create the control plane Linux bond for an isolated network.

     - type: linux_bond
       name: bond_api
       bonding_options: "mode=active-backup"
       use_dhcp: false
       dns_servers:
         get_param: DnsServers
       members:
       - type: interface
         name: nic3
         primary: true
       - type: interface
         name: nic4

   Copy to Clipboard Toggle word wrap

2. Assign VLANs to this Linux bond.

     - type: vlan
       vlan_id:
         get_param: InternalApiNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: InternalApiIpSubnet
   
     - type: vlan
       vlan_id:
         get_param: StorageNetworkVlanID
       device: bond_api
       addresses:
       - ip_netmask:
           get_param: StorageIpSubnet

   Copy to Clipboard Toggle word wrap

3. Set a bridge with a DPDK port to link to the controller.

     - type: ovs_user_bridge
       name: br-link0
       use_dhcp: false
       ovs_extra:
         - str_replace:
             template: set port br-link0 tag=_VLAN_TAG_
             params:
               _VLAN_TAG_:
                  get_param: TenantNetworkVlanID
       addresses:
         - ip_netmask:
             get_param: TenantIpSubnet
       members:
         - type: ovs_dpdk_bond
           name: dpdkbond0
           mtu: 9000
           rx_queue: 2
           members:
             - type: ovs_dpdk_port
               name: dpdk0
               members:
                 - type: interface
                   name: nic7
             - type: ovs_dpdk_port
               name: dpdk1
               members:
                 - type: interface
                   name: nic8

   Copy to Clipboard Toggle word wrap
   Note

   To include multiple DPDK devices, repeat the type code section for each DPDK device you want to add.

   Note

   When using OVS-DPDK, all bridges on the same Compute node should be of type ovs_user_bridge. The director may accept the configuration, but Red Hat OpenStack Platform does not support mixing ovs_bridge and ovs_user_bridge on the same node.