10.9. Using DPDK with SR-IOV

Procedure

1. Map your compute nodes topology to determine which Non-Uniform Memory Access (NUMA) CPUs are isolated for DPDK applications and which ones are reserved for the operating system (OS).

2. If your OpenShift Container Platform cluster uses separate control plane and compute nodes for high-availability:

   1. Label a subset of the compute nodes with a custom role; for example, worker-dpdk:

      $ oc label node <node_name> node-role.kubernetes.io/worker-dpdk=""

   2. Create a new MachineConfigPool manifest that contains the worker-dpdk label in the spec.machineConfigSelector object.

      Example MachineConfigPool manifest:

      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfigPool
      metadata:
        name: worker-dpdk
        labels:
          machineconfiguration.openshift.io/role: worker-dpdk
      spec:
        machineConfigSelector:
          matchExpressions:
            - key: machineconfiguration.openshift.io/role
              operator: In
              values:
                - worker
                - worker-dpdk
        nodeSelector:
          matchLabels:
            node-role.kubernetes.io/worker-dpdk: ""

3. Create a PerformanceProfile manifest that applies to the labeled nodes and the machine config pool that you created in the previous steps. The performance profile specifies the CPUs that are isolated for DPDK applications and the CPUs that are reserved for house keeping.

   Example PerformanceProfile manifest:

   apiVersion: performance.openshift.io/v2
   kind: PerformanceProfile
   metadata:
     name: profile-1
   spec:
     cpu:
       isolated: 4-39,44-79
       reserved: 0-3,40-43
     globallyDisableIrqLoadBalancing: true
     hugepages:
       defaultHugepagesSize: 1G
       pages:
       - count: 8
         node: 0
         size: 1G
     net:
       userLevelNetworking: true
     nodeSelector:
       node-role.kubernetes.io/worker-dpdk: ""
     numa:
       topologyPolicy: single-numa-node

   참고

   The compute nodes automatically restart after you apply the MachineConfigPool and PerformanceProfile manifests.

4. Retrieve the name of the generated RuntimeClass resource from the status.runtimeClass field of the PerformanceProfile object:

   $ oc get performanceprofiles.performance.openshift.io profile-1 -o=jsonpath='{.status.runtimeClass}{"\n"}'

5. Set the previously obtained RuntimeClass name as the default container runtime class for the virt-launcher pods by editing the HyperConverged custom resource (CR):

   $ oc patch hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv \
       --type='json' -p='[{"op": "add", "path": "/spec/defaultRuntimeClass", "value":"<runtimeclass-name>"}]'

   참고

   Editing the HyperConverged CR changes a global setting that affects all VMs that are created after the change is applied.

6. If your DPDK-enabled compute nodes use Simultaneous multithreading (SMT), enable the AlignCPUs enabler by editing the HyperConverged CR:

   $ oc patch hyperconvergeds.v1beta1.hco.kubevirt.io kubevirt-hyperconverged -n openshift-cnv \
       --type='json' -p='[{"op": "replace", "path": "/spec/featureGates/alignCPUs", "value": true}]'

   참고

   Enabling AlignCPUs allows OpenShift Virtualization to request up to two additional dedicated CPUs to bring the total CPU count to an even parity when using emulator thread isolation.

7. Create an SriovNetworkNodePolicy object with the spec.deviceType field set to vfio-pci.

   Example SriovNetworkNodePolicy manifest:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: policy-1
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: intel_nics_dpdk
     deviceType: vfio-pci
     mtu: 9000
     numVfs: 4
     priority: 99
     nicSelector:
       vendor: "8086"
       deviceID: "1572"
       pfNames:
         - eno3
       rootDevices:
         - "0000:19:00.2"
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"