Red Hat Summit Red Hat SummitSupportConsoleDéveloppeursCommencer un essaiContact

Ce contenu n'est pas disponible dans la langue sélectionnée.

Chapter 10. Configuring hardware offloading

As a cluster administrator, you can configure hardware offloading on compatible nodes to increase data processing performance and reduce load on host CPUs.

Before you perform any tasks in the following documentation, ensure that you installed the SR-IOV Network Operator.

10.1. About hardware offloading
Copier lien

Open vSwitch hardware offloading is a method of processing network tasks by diverting them away from the CPU and offloading them to a dedicated processor on a network interface controller. As a result, clusters can benefit from faster data transfer speeds, reduced CPU workloads, and lower computing costs.

The key element for this feature is a modern class of network interface controllers known as SmartNICs. A SmartNIC is a network interface controller that is able to handle computationally-heavy network processing tasks. In the same way that a dedicated graphics card can improve graphics performance, a SmartNIC can improve network performance. In each case, a dedicated processor improves performance for a specific type of processing task.

In OpenShift Container Platform, you can configure hardware offloading for bare metal nodes that have a compatible SmartNIC. Hardware offloading is configured and enabled by the SR-IOV Network Operator.

Hardware offloading is not compatible with all workloads or application types. Only the following two communication types are supported:

  • pod-to-pod
  • pod-to-service, where the service is a ClusterIP service backed by a regular pod

In all cases, hardware offloading takes place only when those pods and services are assigned to nodes that have a compatible SmartNIC. Suppose, for example, that a pod on a node with hardware offloading tries to communicate with a service on a regular node. On the regular node, all the processing takes place in the kernel, so the overall performance of the pod-to-service communication is limited to the maximum performance of that regular node. Hardware offloading is not compatible with DPDK applications.

Enabling hardware offloading on a node, but not configuring pods to use, it can result in decreased throughput performance for pod traffic. You cannot configure hardware offloading for pods that are managed by OpenShift Container Platform.

10.2. Supported devices
Copier lien

Hardware offloading is supported on the following network interface controllers:

Expand
Table 10.1. Supported network interface controllers
ManufacturerModelVendor IDDevice ID

Mellanox

MT27800 Family [ConnectX‑5]

15b3

1017

Mellanox

MT28880 Family [ConnectX‑5 Ex]

15b3

1019

Mellanox

MT2892 Family [ConnectX‑6 Dx]

15b3

101d

Mellanox

MT2894 Family [ConnectX-6 Lx]

15b3

101f

Mellanox

MT42822 BlueField-2 in ConnectX-6 NIC mode

15b3

a2d6

10.3. Prerequisites
Copier lien

10.4. Setting the SR-IOV Network Operator into systemd mode
Copier lien

To support hardware offloading, you must first set the SR-IOV Network Operator into systemd mode.

Prerequisites

  • You installed the OpenShift CLI (oc).
  • You have access to the cluster as a user that has the cluster-admin role.

Procedure

  1. Create a SriovOperatorConfig custom resource (CR) to deploy all the SR-IOV Operator components:

    1. Create a file named sriovOperatorConfig.yaml that contains the following YAML: 

      apiVersion: sriovnetwork.openshift.io/v1
kind: SriovOperatorConfig
metadata:
  name: default
      1 
      
  namespace: openshift-sriov-network-operator
spec:
  enableInjector: true
  enableOperatorWebhook: true
  configurationMode: "systemd"
      2 
      
  logLevel: 2
      Copy to Clipboard Toggle word wrap
      1
      The only valid name for the SriovOperatorConfig resource is default and it must be in the namespace where the Operator is deployed.
      2
      Setting the SR-IOV Network Operator into systemd mode is only relevant for Open vSwitch hardware offloading.

    2. Create the resource by running the following command: 

      $ oc apply -f sriovOperatorConfig.yaml
      Copy to Clipboard Toggle word wrap

To enable hardware offloading, you now create a dedicated machine config pool and configure it to work with the SR-IOV Network Operator.

Prerequisites

  1. SR-IOV Network Operator installed and set into systemd mode.

Procedure

  1. Create a machine config pool for machines you want to use hardware offloading on.

    1. Create a file, such as mcp-offloading.yaml, with content like the following example: 

      apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: mcp-offloading
      1 
      
spec:
  machineConfigSelector:
    matchExpressions:
      - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,mcp-offloading]}
      2 
      
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/mcp-offloading: ""
      3
      Copy to Clipboard Toggle word wrap
      1 2
      The name of your machine config pool for hardware offloading.
      3
      This node role label is used to add nodes to the machine config pool.

    2. Apply the configuration for the machine config pool: 

      $ oc create -f mcp-offloading.yaml
      Copy to Clipboard Toggle word wrap

  2. Add nodes to the machine config pool. Label each node with the node role label of your pool: 

    $ oc label node worker-2 node-role.kubernetes.io/mcp-offloading=""
    Copy to Clipboard Toggle word wrap

  3. Optional: To verify that the new pool is created, run the following command: 

    $ oc get nodes
    Copy to Clipboard Toggle word wrap

    Example output

    NAME       STATUS   ROLES                   AGE   VERSION
master-0   Ready    master                  2d    v1.32.3
master-1   Ready    master                  2d    v1.32.3
worker-0   Ready    worker                  2d    v1.32.3
worker-1   Ready    worker                  2d    v1.32.3
worker-2   Ready    mcp-offloading,worker   47h   v1.32.3
    Copy to Clipboard Toggle word wrap

  4. Add this machine config pool to the SriovNetworkPoolConfig custom resource:

    1. Create a file, such as sriov-pool-config.yaml, with content like the following example: 

      apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkPoolConfig
metadata:
  name: sriovnetworkpoolconfig-offload
  namespace: openshift-sriov-network-operator
spec:
  ovsHardwareOffloadConfig:
    name: mcp-offloading
      1
      Copy to Clipboard Toggle word wrap
      1
      The name of your machine config pool for hardware offloading.

    2. Apply the configuration: 

      $ oc create -f <SriovNetworkPoolConfig_name>.yaml
      Copy to Clipboard Toggle word wrap
      Note

      When you apply the configuration specified in a SriovNetworkPoolConfig object, the SR-IOV Operator drains and restarts the nodes in the machine config pool.

      It might take several minutes for a configuration changes to apply.

10.6. Configuring the SR-IOV network node policy
Copier lien

You can create an SR-IOV network device configuration for a node by creating an SR-IOV network node policy. To enable hardware offloading, you must define the .spec.eSwitchMode field with the value "switchdev".

The following procedure creates an SR-IOV interface for a network interface controller with hardware offloading.

Prerequisites

  • You installed the OpenShift CLI (oc).
  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. Create a file, such as sriov-node-policy.yaml, with content like the following example: 

    apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: sriov-node-policy
    1 
    
  namespace: openshift-sriov-network-operator
spec:
  deviceType: netdevice
    2 
    
  eSwitchMode: "switchdev"
    3 
    
  nicSelector:
    deviceID: "1019"
    rootDevices:
    - 0000:d8:00.0
    vendor: "15b3"
    pfNames:
    - ens8f0
  nodeSelector:
    feature.node.kubernetes.io/network-sriov.capable: "true"
  numVfs: 6
  priority: 5
  resourceName: mlxnics
    Copy to Clipboard Toggle word wrap
    1
    The name for the custom resource object.
    2
    Required. Hardware offloading is not supported with vfio-pci.
    3
    Required.

  2. Apply the configuration for the policy: 

    $ oc create -f sriov-node-policy.yaml
    Copy to Clipboard Toggle word wrap
    Note

    When you apply the configuration specified in a SriovNetworkPoolConfig object, the SR-IOV Operator drains and restarts the nodes in the machine config pool.

    It might take several minutes for a configuration change to apply.

10.6.1. An example SR-IOV network node policy for OpenStack
Copier lien

The following example describes an SR-IOV interface for a network interface controller (NIC) with hardware offloading on Red Hat OpenStack Platform (RHOSP).

An SR-IOV interface for a NIC with hardware offloading on RHOSP

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: ${name}
  namespace: openshift-sriov-network-operator
spec:
  deviceType: switchdev
  isRdma: true
  nicSelector:
    netFilter: openstack/NetworkID:${net_id}
  nodeSelector:
    feature.node.kubernetes.io/network-sriov.capable: 'true'
  numVfs: 1
  priority: 99
  resourceName: ${name}
Copy to Clipboard Toggle word wrap

Follow this procedure to assign a virtual function to the OVN-Kubernetes management port and increase its network traffic performance.

This procedure results in the creation of two pools: the first has a virtual function used by OVN-Kubernetes, and the second comprises the remaining virtual functions.

Prerequisites

  • You installed the OpenShift CLI (oc).
  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. Add the network.operator.openshift.io/smart-nic label to each worker node with a SmartNIC present by running the following command: 

    $ oc label node <node-name> network.operator.openshift.io/smart-nic=
    Copy to Clipboard Toggle word wrap

    Use the oc get nodes command to get a list of the available nodes.

  2. Create a policy named sriov-node-mgmt-vf-policy.yaml for the management port with content such as the following example: 

    apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: sriov-node-mgmt-vf-policy
  namespace: openshift-sriov-network-operator
spec:
  deviceType: netdevice
  eSwitchMode: "switchdev"
  nicSelector:
    deviceID: "1019"
    rootDevices:
    - 0000:d8:00.0
    vendor: "15b3"
    pfNames:
    - ens8f0#0-0
    1 
    
  nodeSelector:
    network.operator.openshift.io/smart-nic: ""
  numVfs: 6
    2 
    
  priority: 5
  resourceName: mgmtvf
    Copy to Clipboard Toggle word wrap
    1
    Replace this device with the appropriate network device for your use case. The #0-0 part of the pfNames value reserves a single virtual function used by OVN-Kubernetes.
    2
    The value provided here is an example. Replace this value with one that meets your requirements. For more information, see SR-IOV network node configuration object in the Additional resources section.

  3. Create a policy named sriov-node-policy.yaml with content such as the following example: 

    apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: sriov-node-policy
  namespace: openshift-sriov-network-operator
spec:
  deviceType: netdevice
  eSwitchMode: "switchdev"
  nicSelector:
    deviceID: "1019"
    rootDevices:
    - 0000:d8:00.0
    vendor: "15b3"
    pfNames:
    - ens8f0#1-5
    1 
    
  nodeSelector:
    network.operator.openshift.io/smart-nic: ""
  numVfs: 6
    2 
    
  priority: 5
  resourceName: mlxnics
    Copy to Clipboard Toggle word wrap
    1
    Replace this device with the appropriate network device for your use case.
    2
    The value provided here is an example. Replace this value with the value specified in the sriov-node-mgmt-vf-policy.yaml file. For more information, see SR-IOV network node configuration object in the Additional resources section.
    Note

    The sriov-node-mgmt-vf-policy.yaml file has different values for the pfNames and resourceName keys than the sriov-node-policy.yaml file.

  4. Apply the configuration for both policies: 

    $ oc create -f sriov-node-policy.yaml
    Copy to Clipboard Toggle word wrap 
    $ oc create -f sriov-node-mgmt-vf-policy.yaml
    Copy to Clipboard Toggle word wrap

  5. Create a Cluster Network Operator (CNO) ConfigMap in the cluster for the management configuration:

    1. Create a ConfigMap named hardware-offload-config.yaml with the following contents: 

      apiVersion: v1
kind: ConfigMap
metadata:
    name: hardware-offload-config
    namespace: openshift-network-operator
data:
    mgmt-port-resource-name: openshift.io/mgmtvf
      Copy to Clipboard Toggle word wrap

    2. Apply the configuration for the ConfigMap: 

      $ oc create -f hardware-offload-config.yaml
      Copy to Clipboard Toggle word wrap

10.8. Creating a network attachment definition
Copier lien

After you define the machine config pool and the SR-IOV network node policy, you can create a network attachment definition for the network interface card you specified.

Prerequisites

  • You installed the OpenShift CLI (oc).
  • You have access to the cluster as a user with the cluster-admin role.

Procedure

  1. Create a file, such as net-attach-def.yaml, with content like the following example: 

    apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-attach-def
    1 
    
  namespace: net-attach-def
    2 
    
  annotations:
    k8s.v1.cni.cncf.io/resourceName: openshift.io/mlxnics
    3 
    
spec:
  config: '{"cniVersion":"0.3.1","name":"ovn-kubernetes","type":"ovn-k8s-cni-overlay","ipam":{},"dns":{}}'
    Copy to Clipboard Toggle word wrap
    1
    The name for your network attachment definition.
    2
    The namespace for your network attachment definition.
    3
    This is the value of the spec.resourceName field you specified in the SriovNetworkNodePolicy object.

  2. Apply the configuration for the network attachment definition: 

    $ oc create -f net-attach-def.yaml
    Copy to Clipboard Toggle word wrap

Verification

  • Run the following command to see whether the new definition is present: 

    $ oc get net-attach-def -A
    Copy to Clipboard Toggle word wrap

    Example output

    NAMESPACE         NAME             AGE
net-attach-def    net-attach-def   43h
    Copy to Clipboard Toggle word wrap

10.9. Adding the network attachment definition to your pods
Copier lien

After you create the machine config pool, the SriovNetworkPoolConfig and SriovNetworkNodePolicy custom resources, and the network attachment definition, you can apply these configurations to your pods by adding the network attachment definition to your pod specifications.

Procedure

  • In the pod specification, add the .metadata.annotations.k8s.v1.cni.cncf.io/networks field and specify the network attachment definition you created for hardware offloading: 

    ....
metadata:
  annotations:
    v1.multus-cni.io/default-network: net-attach-def/net-attach-def
    1
    Copy to Clipboard Toggle word wrap
    1
    The value must be the name and namespace of the network attachment definition you created for hardware offloading.
Retour au début
Red Hat logoGithubredditYoutubeTwitter

Apprendre

Essayez, achetez et vendez

Communautés

À propos de la documentation Red Hat

Nous aidons les utilisateurs de Red Hat à innover et à atteindre leurs objectifs grâce à nos produits et services avec un contenu auquel ils peuvent faire confiance. Découvrez nos récentes mises à jour.

Rendre l’open source plus inclusif

Red Hat s'engage à remplacer le langage problématique dans notre code, notre documentation et nos propriétés Web. Pour plus de détails, consultez le Blog Red Hat.

À propos de Red Hat

Nous proposons des solutions renforcées qui facilitent le travail des entreprises sur plusieurs plates-formes et environnements, du centre de données central à la périphérie du réseau.

Theme

© 2025 Red Hat