Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 10. Example: Configuring OVS-DPDK and SR-IOV with VXLAN tunnelling

This section describes how to deploy Compute nodes with both OVS-DPDK and SR-IOV interfaces. The cluster will be installed with ML2/OVS and VXLAN tunnelling.

Important

You must determine the best values for the OVS-DPDK parameters that you set in the network-environment.yaml file to optimize your OpenStack network for OVS-DPDK. See Deriving DPDK parameters with workflows for details.

10.1. Configuring roles
Copy link

Configure a custom role by copying and editing the default roles_data.yaml file found in /usr/share/openstack-tripleo-heat-templates.

For the purposes of this example, the ComputeOvsDpdkSriov role is created. For information on creating roles in Red Hat OpenStack Platform, see Advanced Overcloud Customization. For details on the specific role used for this example, see roles_data.yaml.

10.2. Configuring OVS-DPDK parameters
Copy link

Important

You must determine the best values for the OVS-DPDK parameters that you set in the network-environment.yaml file to optimize your OpenStack network for OVS-DPDK. See Network Functions Virtualization Planning and Configuration for details.

  1. Add the custom resources for OVS-DPDK under resource_registry: 

      resource_registry:
    # Specify the relative/absolute path to the config files you want to use for override the default.
    OS::TripleO::ComputeOvsDpdkSriov::Net::SoftwareConfig:
      nic-configs/computeovsdpdksriov.yaml
    OS::TripleO::Controller::Net::SoftwareConfig:
      nic-configs/controller.yaml
    Copy to Clipboard Toggle word wrap

  2. Under parameter_defaults, set the tunnel type and network type to vxlan.: 

    NeutronTunnelTypes: 'vxlan'
NeutronNetworkType: 'vxlan'
    Copy to Clipboard Toggle word wrap

  3. Under parameters_defaults, set the bridge mapping: 

    # The OVS logical->physical bridge mappings to use.
NeutronBridgeMappings:
  - dpdk-mgmt:br-link0
    Copy to Clipboard Toggle word wrap

  4. Under parameter_defaults, set the role-specific parameters for the ComputeOvsDpdkSriov role: 

      ##########################
  # OVS DPDK configuration #
  # ########################
  ComputeOvsDpdkSriovParameters:
    KernelArgs: "default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2-19,22-39"
    TunedProfileName: "cpu-partitioning"
    IsolCpusList: "2-19,22-39"
    NovaVcpuPinSet: ['4-19,24-39']
    NovaReservedHostMemory: 2048
    OvsDpdkSocketMemory: "3072,1024"
    OvsDpdkMemoryChannels: "4"
    OvsDpdkCoreList: "0,20,1,21"
    OvsPmdCoreList: "2,22,3,23"
    NovaLibvirtRxQueueSize: 1024
    NovaLibvirtTxQueueSize: 1024
    Copy to Clipboard Toggle word wrap
    Note

    To prevent failures during guest creation, assign at least one CPU with sibling thread on each NUMA node. In the example, the values for the OvsPmdCoreList parameter denote cores 2 and 22 from NUMA 0, and cores 3 and 23 from NUMA 1.

    Note

    Huge pages are consumed by virtual machines, as well as OVS-DPDK using the OvsDpdkSocketMemory parameter. To calculate the number of huge pages available to the virtual machine, subtract the OvsDpdkSocketMemory value from the boot parameter value. You must also add hw:mem_page_size=1GB to the flavor you associate with the DPDK instance.

    Note

    OvsDPDKCoreList and OvsDpdkMemoryChannels are required settings for this procedure and must be set correctly to prevent failures.

  5. Configure the role-specific parameters for SR-IOV: 

      ##########################
  #  SR-IOV configuration  #
  ##########################
  NeutronMechanismDrivers: ['openvswitch','sriovnicswitch']
  NovaSchedulerDefaultFilters: ['RetryFilter','AvailabilityZoneFilter','RamFilter','ComputeFilter','ComputeCapabilitiesFilter','ImagePropertiesFilter','ServerGroupAntiAffinityFilter','ServerGroupAffinityFilter','PciPassthroughFilter','NUMATopologyFilter']
  NovaSchedulerAvailableFilters: ["nova.scheduler.filters.all_filters","nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter"]
  NovaPCIPassthrough:
    - vendor_id: "8086"
      product_id: "1528"
      address: "0000:06:00.0"
      trusted: "true"
      physical_network: "sriov-1"
    - vendor_id: "8086"
      product_id: "1528"
      address: "0000:06:00.1"
      trusted: "true"
      physical_network: "sriov-2"
    Copy to Clipboard Toggle word wrap

10.3. Configuring the Controller node
Copy link

  1. Create the control plane Linux bond for an isolated network. 

      - type: linux_bond
    name: bond_api
    bonding_options: "mode=active-backup"
    use_dhcp: false
    dns_servers:
      get_param: DnsServers
    members:
    - type: interface
      name: nic2
      primary: true
    Copy to Clipboard Toggle word wrap

  2. Assign VLANs to this Linux bond. 

      - type: vlan
    vlan_id:
      get_param: InternalApiNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: InternalApiIpSubnet

  - type: vlan
    vlan_id:
      get_param: StorageNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: StorageIpSubnet

  - type: vlan
    vlan_id:
      get_param: StorageMgmtNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: StorageMgmtIpSubnet

  - type: vlan
    vlan_id:
      get_param: ExternalNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: ExternalIpSubnet
    routes:
    - default: true
      next_hop:
        get_param: ExternalInterfaceDefaultRoute
    Copy to Clipboard Toggle word wrap

  3. Create the OVS bridge for access to neutron-dhcp-agent and neutron-metadata-agent services. 

      - type: ovs_bridge
    name: br-link0
    use_dhcp: false
    mtu: 9000
    members:
    - type: interface
      name: nic3
      mtu: 9000
    - type: vlan
      vlan_id:
        get_param: TenantNetworkVlanID
      mtu: 9000
      addresses:
      - ip_netmask:
          get_param: TenantIpSubnet
    Copy to Clipboard Toggle word wrap

Create the computeovsdpdksriov.yaml file from the default compute.yaml file and make the following changes:

  1. Create the control plane Linux bond for an isolated network. 

      - type: linux_bond
    name: bond_api
    bonding_options: "mode=active-backup"
    use_dhcp: false
    dns_servers:
      get_param: DnsServers
    members:
    - type: interface
      name: nic3
      primary: true
    - type: interface
      name: nic4
    Copy to Clipboard Toggle word wrap

  2. Assign VLANs to this Linux bond. 

      - type: vlan
    vlan_id:
      get_param: InternalApiNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: InternalApiIpSubnet

  - type: vlan
    vlan_id:
      get_param: StorageNetworkVlanID
    device: bond_api
    addresses:
    - ip_netmask:
        get_param: StorageIpSubnet
    Copy to Clipboard Toggle word wrap

  3. Set a bridge with a DPDK port to link to the controller. 

      - type: ovs_user_bridge
    name: br-link0
    use_dhcp: false
    ovs_extra:
      - str_replace:
          template: set port br-link0 tag=_VLAN_TAG_
          params:
            _VLAN_TAG_:
               get_param: TenantNetworkVlanID
    addresses:
      - ip_netmask:
          get_param: TenantIpSubnet
    members:
      - type: ovs_dpdk_bond
        name: dpdkbond0
        mtu: 9000
        rx_queue: 2
        members:
          - type: ovs_dpdk_port
            name: dpdk0
            members:
              - type: interface
                name: nic7
          - type: ovs_dpdk_port
            name: dpdk1
            members:
              - type: interface
                name: nic8
    Copy to Clipboard Toggle word wrap
    Note

    To include multiple DPDK devices, repeat the type code section for each DPDK device you want to add.

    Note

    When using OVS-DPDK, all bridges on the same Compute node should be of type ovs_user_bridge. The director may accept the configuration, but Red Hat OpenStack Platform does not support mixing ovs_bridge and ovs_user_bridge on the same node.

10.5. Deploying the overcloud
Copy link

Run the overcloud_deploy.sh script to deploy the overcloud. 

#!/bin/bash

openstack overcloud deploy \
--templates \
-r /home/stack/ospd-13-vxlan-dpdk-sriov-ctlplane-dataplane-bonding-hybrid/roles_data.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/host-config-and-reboot.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dpdk.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-sriov.yaml \
-e /home/stack/ospd-13-vxlan-dpdk-sriov-ctlplane-dataplane-bonding-hybrid/ovs-dpdk-permissions.yaml \
-e /home/stack/ospd-13-vxlan-dpdk-sriov-ctlplane-dataplane-bonding-hybrid/overcloud_images.yaml \
-e /home/stack/ospd-13-vxlan-dpdk-sriov-ctlplane-dataplane-bonding-hybrid/network-environment.yaml \
--log-file overcloud_install.log &> overcloud_install.log
Copy to Clipboard Toggle word wrap
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat