Red Hat Summit 红帽全球峰会支持控制台（Console）开发人员开始试用联系人

此内容没有您所选择的语言版本。

Chapter 8. Configuring an OVS-DPDK deployment

This section deploys DPDK with Open vSwitch (OVS-DPDK) within the Red Hat OpenStack Platform environment. The overcloud usually consists of nodes in predefined roles such as Controller nodes, Compute nodes, and different storage node types. Each of these default roles contains a set of services defined in the core Heat templates on the director node.

You must install and configure the undercloud before you can deploy the overcloud. See the Director Installation and Usage Guide for details.

Important

You must determine the best values for the OVS-DPDK parameters that you set in the network-environment.yaml file to optimize your OpenStack network for OVS-DPDK.

Note

Do not edit or change isolated_cores or other values in etc/tuned/cpu-partitioning-variables.conf that are modified by these director Heat templates.

8.1. Deriving DPDK parameters with workflows
复制链接

Important

This feature is available in this release as a Technology Preview, and therefore is not fully supported by Red Hat. It should only be used for testing, and should not be deployed in a production environment. For more information about Technology Preview features, see Scope of Coverage Details.

See Section 7.2, “Overview of workflows and derived parameters” for an overview of the Mistral workflow for DPDK.

Prerequisites

You must have Bare Metal introspection, including hardware inspection extras (inspection_extras) enabled to provide the data retrieved by this workflow. Hardware inspection extras are enabled by default. See Inspecting the Hardware of Nodes.

Define the Workflows and Input Parameters for DPDK

The following lists the input parameters you can provide to the OVS-DPDK workflows:

num_phy_cores_per_numa_node_for_pmd
This input parameter specifies the required minimum number of cores for the NUMA node associated with the DPDK NIC. One physical core is assigned for the other NUMA nodes not associated with DPDK NIC. Ensure that this parameter is set to 1.
huge_page_allocation_percentage
This input parameter specifies the required percentage of total memory (excluding NovaReservedHostMemory) that can be configured as huge pages. The KernelArgs parameter is derived using the calculated huge pages based on the huge_page_allocation_percentage specified. Ensure that this parameter is set to 50.

The workflows use these input parameters along with the bare-metal introspection details to calculate appropriate DPDK parameter values.

To define the workflows and input parameters for DPDK:

  1. Copy the /usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-params.yaml file to a local directory and set the input parameters to suit your environment. 

      workflow_parameters:
    tripleo.derive_params.v1.derive_parameters:
      # DPDK Parameters # DPDK Parameters # DPDK Parameters #
      # Specifices the minimum number of CPU physical cores to be allocated for DPDK
      # PMD threads. The actual allocation will be based on network config, if
      # the a DPDK port is associated with a numa node, then this configuration
      # will be used, else 1.
      num_phy_cores_per_numa_node_for_pmd: 1
      # Amount of memory to be configured as huge pages in percentage. Ouf the
      # total available memory (excluding the NovaReservedHostMemory), the
      # specified percentage of the remaining is configured as huge pages.
      huge_page_allocation_percentage: 50
    Copy to Clipboard Toggle word wrap

  2. Run the openstack overcloud deploy command and include the following:

    • The update-plan-only option
    • The role file and all environment files specific to your environment

    • The plan-environment-derived-parms.yaml file with the --plan-environment-file optional argument 

      $ openstack overcloud deploy --templates --update-plan-only \
-r /home/stack/roles_data.yaml \
-e /home/stack/<environment-file> \
... #repeat as necessary ...
-p /home/stack/plan-environment-derived-params.yaml
      Copy to Clipboard Toggle word wrap

The output of this command shows the derived results, which are also merged into the plan-environment.yaml file. 

Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: 55ba73f2-2ef4-4da1-94e9-eae2fdc35535
Waiting for messages on queue 472a4180-e91b-4f9e-bd4c-1fbdfbcf414f with no timeout.
Removing the current plan files
Uploading new plan files
Started Mistral Workflow tripleo.plan_management.v1.update_deployment_plan. Execution ID: 7fa995f3-7e0f-4c9e-9234-dd5292e8c722
Plan updated.
Processing templates in the directory /tmp/tripleoclient-SY6RcY/tripleo-heat-templates
Invoking workflow (tripleo.derive_params.v1.derive_parameters) specified in plan-environment file
Started Mistral Workflow tripleo.derive_params.v1.derive_parameters. Execution ID: 2d4572bf-4c5b-41f8-8981-c84a363dd95b
Workflow execution is completed. result:
ComputeOvsDpdkParameters:
 IsolCpusList: 1-7,17-23,9-15,25-31
 KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-7,17-23,9-15,25-31
 NovaReservedHostMemory: 4096
 NovaVcpuPinSet: 2-7,18-23,10-15,26-31
 OvsDpdkCoreList: 0,16,8,24
 OvsDpdkMemoryChannels: 4
 OvsDpdkSocketMemory: 1024,1024
 OvsPmdCoreList: 1,17,9,25
Copy to Clipboard Toggle word wrap
Note

The OvsDpdkMemoryChannels parameter cannot be derived from introspection details. In most cases, this value should be 4.

Deploy the Overcloud with the Derived Parameters

To deploy the overcloud with these derived parameters:

  1. Copy the derived parameters from the deploy command output to the network-environment.yaml file. 

      # DPDK compute node.
  ComputeOvsDpdkParameters:
    KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-7,17-23,9-15,25-31
    TunedProfileName: "cpu-partitioning"
    IsolCpusList: "1-7,17-23,9-15,25-31"
    NovaVcpuPinSet: ['2-7,18-23,10-15,26-31']
    NovaReservedHostMemory: 4096
    OvsDpdkSocketMemory: "1024,1024"
    OvsDpdkMemoryChannels: "4"
    OvsDpdkCoreList: "0,16,8,24"
    OvsPmdCoreList: "1,17,9,25"
    Copy to Clipboard Toggle word wrap
    Note

    You must assign at least one CPU (with sibling thread) on each NUMA node with or without DPDK NICs present for DPDK PMD to avoid failures in creating guest instances.

    Note

    These parameters apply exclusively to the role ComputeOvsDpdk, and will not apply to other roles, including Compute or ComputeSriov that may exist on the same cluster. You can apply these parameters globally, but any global parameters are overwritten by role-specific parameters.

  2. Deploy the overcloud using the role file and all environment files specific to your environment. See Deploying the Overcloud for details.

8.2. OVS-DPDK topology
复制链接

With Red Hat OpenStack Platform, you can create custom deployment roles, using the composable roles feature, adding or removing services from each role. For more information on Composable Roles, see Composable Roles and Services.

This image shows a sample Open vSwitch with Data Plane Development Kit (OVS-DPDK) topology with two bonded ports for the control plane and data plane:

OpenStack NFV Config Guide Topology 450694 0617 ECE OVS DPDK
View larger image
OpenStack NFV Config Guide Topology 450694 0617 ECE OVS DPDK

Configuring OVS-DPDK comprises the following tasks:

  • If you use composable roles, copy and modify the roles_data.yaml file to add the custom role for OVS-DPDK.
  • Update the appropriate network-environment.yaml file to include parameters for kernel arguments and DPDK arguments.
  • Update the compute.yaml file to include the bridge for DPDK interface parameters.
  • Update the controller.yaml file to include the same bridge details for DPDK interface parameters.
  • Run the overcloud_deploy.sh script to deploy the overcloud with the DPDK parameters.
Note

This guide provides examples for CPU assignments, memory allocation, and NIC configurations that can vary from your topology and use case. See the Network Functions Virtualization Product Guide and Chapter 2, Hardware requirements to understand the hardware and configuration options.

Prerequisites

Note

The Red Hat OpenStack Platform operates in OVS client mode for OVS-DPDK deployments.

8.3. Setting the MTU value for OVS-DPDK interfaces
复制链接

Red Hat OpenStack Platform supports jumbo frames for Open vSwitch with Data Plane Development Kit (OVS-DPDK). To set the maximum transmission unit (MTU) value for jumbo frames you must:

  • Set the global MTU value for networking in the network-environment.yaml file.
  • Set the physical DPDK port MTU value in the compute.yaml file. This value is also used by the vhost user interface.
  • Set the MTU value within any guest instances on the Compute node to ensure that you have a comparable MTU value from end to end in your configuration.
Note

VXLAN packets include an extra 50 bytes in the header. Calculate your MTU requirements based on these additional header bytes. For example, an MTU value of 9000 means the VXLAN tunnel MTU value is 8950 to account for these extra bytes.

Note

You do not need any special configuration for the physical NIC because the NIC is controlled by the DPDK PMD and has the same MTU value set by the compute.yaml file. You cannot set an MTU value larger than the maximum value supported by the physical NIC.

To set the MTU value for OVS-DPDK interfaces:

  1. Set the NeutronGlobalPhysnetMtu parameter in the network-environment.yaml file. 

    parameter_defaults:
  # MTU global configuration
  NeutronGlobalPhysnetMtu: 9000
    Copy to Clipboard Toggle word wrap
    Note

    Ensure that the NeutronDpdkSocketMemory value in the network-environment.yaml file is large enough to support jumbo frames. See Section 7.4.2, “Memory parameters” for details.

  2. Set the MTU value on the bridge to the Compute node in the controller.yaml file. 

      -
    type: ovs_bridge
    name: br-link0
    use_dhcp: false
    members:
      -
        type: interface
        name: nic3
        mtu: 9000
    Copy to Clipboard Toggle word wrap

  3. Set the MTU values for an OVS-DPDK bond in the compute.yaml file: 

    - type: ovs_user_bridge
  name: br-link0
  use_dhcp: false
  members:
    - type: ovs_dpdk_bond
      name: dpdkbond0
      mtu: 9000
      rx_queue: 2
      members:
        - type: ovs_dpdk_port
          name: dpdk0
          mtu: 9000
          members:
            - type: interface
              name: nic4
        - type: ovs_dpdk_port
          name: dpdk1
          mtu: 9000
          members:
            - type: interface
              name: nic5
    Copy to Clipboard Toggle word wrap

8.4. Configuring a firewall for security groups
复制链接

Dataplane interfaces need a high degree of performance in a stateful firewall. To protect these interfaces, consider deploying a telco grade firewall as a virtual network function (VNF).

Controlplane interfaces can be configured by setting the NeutronOVSFirewallDriver parameter to openvswitch. This configures OpenStack Networking to use the flow-based OVS firewall driver. This is set in the network-environment.yaml file under parameter_defaults.

Example: 

parameter_defaults:
  NeutronOVSFirewallDriver: openvswitch
Copy to Clipboard Toggle word wrap

When the OVS firewall driver is used, it is important to disable it for dataplane interfaces. This can be done with the openstack port set command.

Example: 

openstack port set --no-security-group  --disable-port-security ${PORT}
Copy to Clipboard Toggle word wrap

8.5. Setting multiqueue for OVS-DPDK interfaces
复制链接

To set set same number of queues for interfaces in Open vSwitch with Data Plane Development Kit (OVS-DPDK) on the Compute node, modify the compute.yaml file as follows: 

- type: ovs_user_bridge
  name: br-link0
  use_dhcp: false
  members:
    - type: ovs_dpdk_bond
      name: dpdkbond0
      mtu: 9000
      rx_queue: 2
      members:
        - type: ovs_dpdk_port
          name: dpdk0
          mtu: 9000
          members:
            - type: interface
              name: nic4
        - type: ovs_dpdk_port
          name: dpdk1
          mtu: 9000
          members:
            - type: interface
              name: nic5
Copy to Clipboard Toggle word wrap

8.6. Deploying the overcloud
复制链接

  1. Assign at least one CPU (with sibling thread) on each NUMA node with or without DPDK NICs present for DPDK PMD to avoid failures in creating guest instances.

  2. Ensure parameters for your DPDK compute role are populated in network-environment.yaml. These can be copied from derived OVS-DPDK parameters if needed: 

     # DPDK compute node.
  ComputeOvsDpdkParameters:
    KernelArgs: default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-7,17-23,9-15,25-31
    TunedProfileName: "cpu-partitioning"
    IsolCpusList: "1-7,17-23,9-15,25-31"
    NovaVcpuPinSet: ['2-7,18-23,10-15,26-31']
    NovaReservedHostMemory: 4096
    OvsDpdkSocketMemory: "1024,1024"
    OvsDpdkMemoryChannels: "4"
    OvsDpdkCoreList: "0,16,8,24"
    OvsPmdCoreList: "1,17,9,25"
    Copy to Clipboard Toggle word wrap

  3. Deploy the overcloud using the openstack overcloud deploy command.

    • Include the role file and all environment files specific to your environment.

    • Apply the KernelArgs and TunedProfile parameters by including the host-config-and-reboot.yaml file from /usr/share/openstack-tripleo-heat-templates/environments to your deployment script: 

      TEMPLATES_HOME=”/usr/share/openstack-tripleo-heat-templates”
CUSTOM_TEMPLATES=”/home/stack/templates”

openstack overcloud deploy --templates \
-r ${CUSTOM_TEMPLATES}/roles_data.yaml \
-e ${TEMPLATES_HOME}/environments/host-config-and-reboot.yaml \
-e ${CUSTOM_TEMPLATES}/network-environment.yaml \
-e ${CUSTOM_TEMPLATES}/controller.yaml
-e ${CUSTOM_TEMPLATES}/computeovsdpdk.yaml \
...
      Copy to Clipboard Toggle word wrap

8.7. Known limitations
复制链接

There are certain limitations when configuring OVS-DPDK with Red Hat OpenStack Platform for the NFV use case:

  • Use Linux bonds for control plane networks. Ensure that both PCI devices used in the bond are on the same NUMA node for optimum performance. Neutron Linux bridge configuration is not supported by Red Hat.
  • Huge pages are required for every instance running on the hosts with OVS-DPDK. If huge pages are not present in the guest, the interface appears but does not function.
  • With OVS-DPDK, there is a performance degradation of services that use tap devices, such as Distributed Virtual Routing (DVR). The resulting performance is not suitable for a production environment.
  • When using OVS-DPDK, all bridges on the same Compute node must be of type ovs_user_bridge. The director may accept the configuration, but Red Hat OpenStack Platform does not support mixing ovs_bridge and ovs_user_bridge on the same node.

After you have completed configuring Open vSwitch with Data Plane Development Kit (OVS-DPDK) for your Red Hat OpenStack Platform deployment with NFV, you can create a flavor and deploy an instance with the following steps:

  1. Create an aggregate group and add relevant hosts for OVS-DPDK. Define metadata, for example dpdk=true, that matches defined flavor metadata. 

     # openstack aggregate create dpdk_group
 # openstack aggregate add host dpdk_group [compute-host]
 # openstack aggregate set --property dpdk=true dpdk_group
    Copy to Clipboard Toggle word wrap
    Note

    Use host aggregates to separate CPU pinned instances from unpinned instances. Instances that do not use CPU pinning do not respect the resourcing requirements of instances that use CPU pinning.

  2. Create a flavor. 

    # openstack flavor create <flavor> --ram <MB> --disk <GB> --vcpus <#>
    Copy to Clipboard Toggle word wrap

  3. Set additional flavor properties. Note that the defined metadata, dpdk=true, matches the defined metadata in the DPDK aggregate. 

    # openstack flavor set <flavor> --property dpdk=true --property hw:cpu_policy=dedicated --property hw:mem_page_size=1GB --property hw:emulator_threads_policy=isolate
    Copy to Clipboard Toggle word wrap

    In this example, m1.medium_huge_4cpu is the flavor name and the remaining parameters set the other properties for the flavor.

See Configure Emulator Threads to run on a Dedicated Physical CPU for details on the emulator threads policy for performance improvements.

  1. Create the network. 

    # openstack network create net1 --provider-physical-network tenant --provider-network-type vlan --provider-segment <VLAN-ID>
    Copy to Clipboard Toggle word wrap

  2. Deploy an instance. 

    # openstack server create --flavor <flavor> --image <glance image> --nic net-id=<network ID>
    Copy to Clipboard Toggle word wrap

For using multi-queue with OVS-DPDK, set the hw_vif_multiqueue_enabled property on an image then set the hw.vif_multiqueue_enabled property on a flavor:

  1. Set the image properties. 

    # openstack image set --property hw_vif_multiqueue_enabled=true <image>
    Copy to Clipboard Toggle word wrap

  2. Set additional flavor properties. 

    # openstack flavor set --property hw:vif_multiqueue_enabled=true <flavor>
    Copy to Clipboard Toggle word wrap

8.9. Troubleshooting the configuration
复制链接

This section describes the steps to troubleshoot the Open vSwitch with Data Plane Development Kit (DPDK-OVS) configuration.

  1. Review the bridge configuration and confirm that the bridge was created with the datapath_type=netdev. 

    # ovs-vsctl list bridge br0
_uuid               : bdce0825-e263-4d15-b256-f01222df96f3
auto_attach         : []
controller          : []
datapath_id         : "00002608cebd154d"
datapath_type       : netdev
datapath_version    : "<built-in>"
external_ids        : {}
fail_mode           : []
flood_vlans         : []
flow_tables         : {}
ipfix               : []
mcast_snooping_enable: false
mirrors             : []
name                : "br0"
netflow             : []
other_config        : {}
ports               : [52725b91-de7f-41e7-bb49-3b7e50354138]
protocols           : []
rstp_enable         : false
rstp_status         : {}
sflow               : []
status              : {}
stp_enable          : false
    Copy to Clipboard Toggle word wrap

  2. Confirm that the docker container neutron_ovs_agent is configured to start automatically. 

    # docker inspect neutron_ovs_agent | grep -A1 RestartPolicy
            "RestartPolicy": {
                "Name": "always",
    Copy to Clipboard Toggle word wrap

  3. If the container is having trouble starting, you can view any related messages. 

    # less /var/log/containers/neutron/openvswitch-agent.log
    Copy to Clipboard Toggle word wrap

  4. Confirm that the PMD CPU mask of the ovs-dpdk are pinned to the CPUs. In case of HT, use sibling CPUs.

    For example, take CPU4: 

    # cat /sys/devices/system/cpu/cpu4/topology/thread_siblings_list
4,20
    Copy to Clipboard Toggle word wrap

    So, using CPU 4 and 20: 

    # ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x100010
    Copy to Clipboard Toggle word wrap

    Display their status: 

    # tuna -t ovs-vswitchd -CP
thread  ctxt_switches pid SCHED_ rtpri affinity voluntary nonvoluntary       cmd
3161	OTHER 	0    	6	765023      	614	ovs-vswitchd
3219   OTHER 	0    	6     	1        	0   	handler24
3220   OTHER 	0    	6     	1        	0   	handler21
3221   OTHER 	0    	6     	1        	0   	handler22
3222   OTHER 	0    	6     	1        	0   	handler23
3223   OTHER 	0    	6     	1        	0   	handler25
3224   OTHER 	0    	6     	1        	0   	handler26
3225   OTHER 	0    	6     	1        	0   	handler27
3226   OTHER 	0    	6     	1        	0   	handler28
3227   OTHER 	0    	6     	2        	0   	handler31
3228   OTHER 	0    	6     	2        	4   	handler30
3229   OTHER 	0    	6     	2        	5   	handler32
3230   OTHER 	0    	6	953538      	431   revalidator29
3231   OTHER 	0    	6   1424258      	976   revalidator33
3232   OTHER 	0    	6   1424693      	836   revalidator34
3233   OTHER 	0    	6	951678      	503   revalidator36
3234   OTHER 	0    	6   1425128      	498   revalidator35
*3235   OTHER 	0    	4	151123       	51       	pmd37*
*3236   OTHER 	0   	20	298967       	48       	pmd38*
3164   OTHER 	0    	6 	47575        	0  dpdk_watchdog3
3165   OTHER 	0    	6	237634        	0   vhost_thread1
3166   OTHER 	0    	6  	3665        	0       	urcu2
    Copy to Clipboard Toggle word wrap
返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务，以及可以信赖的内容，帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情，请参阅红帽博客.

關於紅帽

我们提供强化的解决方案，使企业能够更轻松地跨平台和环境（从核心数据中心到网络边缘）工作。

Theme

© 2025 Red Hat