10.8. Connecting a virtual machine to an SR-IOV network
You can connect a virtual machine (VM) to the physical network by using a Single Root I/O Virtualization (SR-IOV) device.
To configure the SR-IOV network and attach the VM to that network, perform the following steps:
- Configure an SR-IOV physical network device.
- Define the secondary SR-IOV network.
- Attach the VM to the SR-IOV network.
10.8.1. Configuring SR-IOV network devices 링크 복사링크가 클립보드에 복사되었습니다!
The SR-IOV Network Operator adds the SriovNetworkNodePolicy.sriovnetwork.openshift.io custom resource definition (CRD) to OpenShift Container Platform. You can configure an SR-IOV network device by creating a SriovNetworkNodePolicy custom resource (CR).
When applying the configuration specified in a SriovNetworkNodePolicy CR, the SR-IOV Operator might drain the nodes, and in some cases, reboot nodes. Reboot only happens in the following cases:
-
With Mellanox NICs (
mlx5driver) a node reboot happens every time the number of virtual functions (VFs) increase on a physical function (PF). -
With Intel NICs, a reboot only happens if the kernel parameters do not include
intel_iommu=onandiommu=pt.
It might take several minutes for a configuration change to apply.
Prerequisites
-
You installed the OpenShift CLI (
oc). -
You have access to the cluster as a user with the
cluster-adminrole. - You have installed the SR-IOV Network Operator.
- You have enough available nodes in your cluster to handle the evicted workload from drained nodes.
- You have not selected any control plane nodes for SR-IOV network device configuration.
Procedure
Create an
SriovNetworkNodePolicyobject, and then save the YAML in the<name>-sriov-node-network.yamlfile. Replace<name>with the name for this configuration.apiVersion: sriovnetwork.openshift.io/v1 kind: SriovNetworkNodePolicy metadata: name: <name> namespace: openshift-sriov-network-operator spec: resourceName: <sriov_resource_name> nodeSelector: feature.node.kubernetes.io/network-sriov.capable: "true" priority: <priority> mtu: <mtu> numVfs: <num> nicSelector: vendor: "<vendor_code>" deviceID: "<device_id>" pfNames: ["<pf_name>", ...] rootDevices: ["<pci_bus_id>", "..."] deviceType: vfio-pci isRdma: false-
metadata.namedefines a name for theSriovNetworkNodePolicyobject. -
metadata.namespacedefines the namespace where the SR-IOV Network Operator is installed. -
spec.resourceNamedefines the resource name of the SR-IOV device plugin. You can create multipleSriovNetworkNodePolicyobjects for a resource name. -
spec.nodeSelector.feature.node.kubernetes.io/network-sriov.capabledefines the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes. -
spec.priorityis an optional field that defines an integer value between0and99. A smaller number gets higher priority, so a priority of10is higher than a priority of99. The default value is99. -
spec.mtuis an optional field that defines a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models. -
spec.numVfsdefines the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than127. spec.nicSelectordefines the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters.참고It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally. If you specify
rootDevices, you must also specify a value forvendor,deviceID, orpfNames.If you specify both
pfNamesandrootDevicesat the same time, ensure that they point to an identical device.-
spec.nicSelector.vendoris an optional field that defines the vendor hex code of the SR-IOV network device. The only allowed values are either8086or15b3. -
spec.nicSelector.deviceIDis an optional field that defines the device hex code of SR-IOV network device. The only allowed values are158b,1015,1017. -
spec.nicSelector.pfNamesis an optional field that defines an array of one or more physical function (PF) names for the Ethernet device. -
spec.nicSelector.rootDevicesis an optional field that defines an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format:0000:02:00.1. -
spec.deviceTypedefines the driver type. Thevfio-pcidriver type is required for virtual functions in OpenShift Virtualization. spec.isRdmais an optional field that defines whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, setisRdmatofalse. The default value isfalse.참고If
isRDMAflag is set totrue, you can continue to use the RDMA enabled VF as a normal network device. A device can be used in either mode.
-
-
Optional: Label the SR-IOV capable cluster nodes with
SriovNetworkNodePolicy.Spec.NodeSelectorif they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes". Create the
SriovNetworkNodePolicyobject. When running the following command, replace<name>with the name for this configuration:$ oc create -f <name>-sriov-node-network.yamlAfter applying the configuration update, all the pods in the
sriov-network-operatornamespace change to theRunningstatus.To verify your SR-IOV network device configuration, enter the following command and replace
<node_name>with the name of the node where you configured the device.$ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'