Chapter 11. Networking

Procedure

1. Edit the interfaces spec of your virtual machine configuration file:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: default
                 masquerade: {} 

   1

   
                 ports: 

   2

   
                   - port: 80
   # ...
         networks:
         - name: default
           pod: {}

   Copy to Clipboard Toggle word wrap

   1

   Connect using masquerade mode.

   2

   Optional: List the ports that you want to expose from the virtual machine, each specified by the port field. The port value must be a number between 0 and 65536. When the ports array is not used, all ports in the valid range are open to incoming traffic. In this example, incoming traffic is allowed on port 80.

   Note

   Ports 49152 and 49153 are reserved for use by the libvirt platform and all other incoming traffic to these ports is dropped.

2. Create the virtual machine:

   $ oc create -f <vm-name>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. In a new virtual machine configuration, include an interface with masquerade and configure the IPv6 address and default gateway by using cloud-init.

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm-ipv6
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: default
                 masquerade: {} 

   1

   
                 ports:
                   - port: 80 

   2

   
   # ...
         networks:
         - name: default
           pod: {}
         volumes:
         - cloudInitNoCloud:
             networkData: |
               version: 2
               ethernets:
                 eth0:
                   dhcp4: true
                   addresses: [ fd10:0:2::2/120 ] 

   3

   
                   gateway6: fd10:0:2::1 

   4

   Copy to Clipboard Toggle word wrap

   1

   Connect using masquerade mode.

   2

   Allows incoming traffic on port 80 to the virtual machine.

   3

   The static IPv6 address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::2/120.

   4

   The gateway IP address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::1.

2. Create the virtual machine in the namespace:

   $ oc create -f example-vm-ipv6.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a NodeNetworkConfigurationPolicy object to map the OVN-Kubernetes secondary network to an Open vSwitch (OVS) bridge:

   Example NodeNetworkConfigurationPolicy manifest

   apiVersion: nmstate.io/v1
   kind: NodeNetworkConfigurationPolicy
   metadata:
     name: mapping 

   1

   
   spec:
     nodeSelector:
       node-role.kubernetes.io/worker: '' 

   2

   
     desiredState:
       ovn:
         bridge-mappings:
         - localnet: localnet1 

   3

   
           bridge: br-ex 

   4

   
           state: present 

   5

   Copy to Clipboard Toggle word wrap

   1

   The name of the configuration object.

   2

   Specifies the nodes to which the node network configuration policy is applied. The recommended node selector value is node-role.kubernetes.io/worker: ''.

   3

   The name of the additional network from which traffic is forwarded to the OVS bridge. This attribute must match the value of the spec.network.localnet.physicalNetworkName field of the ClusterUserDefinedNetwork object that defines the OVN-Kubernetes additional network. This example uses the name localnet1.

   4

   The name of the OVS bridge on the node. This value is required if the state attribute is present or not specified.

   5

   The state of the mapping. Must be either present to add the mapping or absent to remove the mapping. The default value is present.

   Important

   OpenShift Virtualization does not support Linux bridge bonding modes 0, 5, and 6. For more information, see Which bonding modes work when used with a bridge that virtual machine guests or containers connect to?.

2. Apply the NodeNetworkConfigurationPolicy manifest by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your NodeNetworkConfigurationPolicy manifest YAML file.

3. Create a ClusterUserDefinedNetwork object to create a localnet secondary network:

   Example ClusterUserDefinedNetwork manifest

   apiVersion: k8s.ovn.org/v1
   kind: ClusterUserDefinedNetwork
   metadata:
     name: cudn-localnet 

   1

   
   spec:
     namespaceSelector: 

   2

   
       matchExpressions: 

   3

   
       - key: kubernetes.io/metadata.name
         operator: In 

   4

   
         values: ["red", "blue"]
     network:
       topology: Localnet 

   5

   
       localnet:
           role: Secondary 

   6

   
           physicalNetworkName: localnet1 

   7

   
           ipam:
             mode: Disabled 

   8

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   The name of the ClusterUserDefinedNetwork custom resource.

   2

   The set of namespaces that the cluster UDN applies to. The namespace selector must not point to the following values: default; an openshift-* namespace; or any global namespaces that are defined by the Cluster Network Operator (CNO).

   3

   The type of selector. In this example, the matchExpressions selector selects objects that have the label kubernetes.io/metadata.name with the value red or blue.

   4

   The type of operator. Possible values are In, NotIn, and Exists.

   5

   The topological configuration of the network. A Localnet topology connects the logical network to the physical underlay.

   6

   Specifies whether the UDN is primary or secondary. The required value is Secondary for topology: Localnet.

   7

   The name of the OVN-Kubernetes bridge mapping that is configured on the node. This value must match the spec.desiredState.ovn.bridge-mappings.localnet field in the NodeNetworkConfigurationPolicy manifest that you previously created. This ensures that you are bridging to the intended segment of your physical network.

   8

   Specifies whether IP address management (IPAM) is enabled or disabled. The required value is Disabled. OpenShift Virtualization does not support configuring IPAM for virtual machines.

4. Apply the ClusterUserDefinedNetwork manifest by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your ClusterUserDefinedNetwork manifest YAML file.

Procedure

1. Create a Namespace object similar to the following example:

   Example Namespace manifest

   apiVersion: v1
   kind: Namespace
   metadata:
     name: red
   # ...

   Copy to Clipboard Toggle word wrap

2. Apply the Namespace manifest by running the following command:

   oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your Namespace manifest YAML file.

Procedure

1. Edit the VirtualMachine manifest to add the CUDN interface details, as in the following example:

   Example VirtualMachine manifest

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
     namespace: red 

   1

   
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: secondary_localnet 

   2

   
                 bridge: {}
           machine:
             type: ""
           resources:
             requests:
               memory: 2048M
         networks:
         - name: secondary_localnet 

   3

   
           multus:
             networkName: <localnet_cudn_name> 

   4

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   The namespace in which the VM is located. This value must match a namespace that is associated with the secondary CUDN.

   2

   The name of the secondary user-defined network interface.

   3

   The name of the network. This must match the value of the spec.template.spec.domain.devices.interfaces.name field.

   4

   The name of the localnet ClusterUserDefinedNetwork object that you previously created.

2. Apply the VirtualMachine manifest by running the following command:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your VirtualMachine manifest YAML file.

Procedure

1. Edit the VirtualMachine manifest to add the label for service creation:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
     namespace: example-namespace
   spec:
     runStrategy: Halted
     template:
       metadata:
         labels:
           special: key 

   1

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Add special: key to the spec.template.metadata.labels stanza.

   Note

   Labels on a virtual machine are passed through to the pod. The special: key label must match the label in the spec.selector attribute of the Service manifest.

2. Save the VirtualMachine manifest file to apply your changes.

3. Create a Service manifest to expose the VM:

   apiVersion: v1
   kind: Service
   metadata:
     name: example-service
     namespace: example-namespace
   spec:
   # ...
     selector:
       special: key 

   1

   
     type: NodePort 

   2

   
     ports: 

   3

   
       protocol: TCP
       port: 80
       targetPort: 9376
       nodePort: 30000

   Copy to Clipboard Toggle word wrap

   1

   Specify the label that you added to the spec.template.metadata.labels stanza of the VirtualMachine manifest.

   2

   Specify ClusterIP, NodePort, or LoadBalancer.

   3

   Specifies a collection of network ports and protocols that you want to expose from the virtual machine.

4. Save the Service manifest file.

5. Create the service by running the following command:

   $ oc create -f example-service.yaml

   Copy to Clipboard Toggle word wrap
6. Restart the VM to apply the changes.

Procedure

1. Create a Service manifest to expose the VM, such as the following example:

   apiVersion: v1
   kind: Service
   metadata:
     name: mysubdomain 

   1

   
   spec:
     selector:
       expose: me 

   2

   
     clusterIP: None 

   3

   
     ports: 

   4

   
     - protocol: TCP
       port: 1234
       targetPort: 1234

   Copy to Clipboard Toggle word wrap

   1

   The name of the service. This must match the spec.subdomain attribute in the VirtualMachine manifest file.

   2

   This service selector must match the expose:me label in the VirtualMachine manifest file.

   3

   Specifies a headless service.

   4

   The list of ports that are exposed by the service. You must define at least one port. This can be any arbitrary value as it does not affect the headless service.

2. Save the Service manifest file.

3. Create the service by running the following command:

   $ oc create -f headless_service.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Edit the VirtualMachine manifest to add the service selector label and subdomain by running the following command:

   $ oc edit vm <vm_name>

   Copy to Clipboard Toggle word wrap

   Example VirtualMachine manifest file

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: vm-fedora
   spec:
     template:
       metadata:
         labels:
           expose: me 

   1

   
       spec:
         hostname: "myvm" 

   2

   
         subdomain: "mysubdomain" 

   3

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   The expose:me label must match the spec.selector attribute of the Service manifest that you previously created.

   2

   If this attribute is not specified, the resulting DNS A record takes the form of <vm.metadata.name>.<vm.spec.subdomain>.<vm.metadata.namespace>.svc.cluster.local.

   3

   The spec.subdomain attribute must match the metadata.name value of the Service object.

2. Save your changes and exit the editor.
3. Restart the VM to apply the changes.

Procedure

1. Connect to the VM console by entering the following command:

   $ virtctl console vm-fedora

   Copy to Clipboard Toggle word wrap

2. To connect to the VM by using the requested FQDN, run the following command:

   $ ping myvm.mysubdomain.<namespace>.svc.cluster.local

   Copy to Clipboard Toggle word wrap

   Example output

   PING myvm.mysubdomain.default.svc.cluster.local (10.244.0.57) 56(84) bytes of data.
   64 bytes from myvm.mysubdomain.default.svc.cluster.local (10.244.0.57): icmp_seq=1 ttl=64 time=0.029 ms

   Copy to Clipboard Toggle word wrap

   In the preceding example, the DNS entry for myvm.mysubdomain.default.svc.cluster.local points to 10.244.0.57, which is the cluster IP address that is currently assigned to the VM.

Procedure

1. Create an SriovNetworkNodePolicy object, and then save the YAML in the <name>-sriov-node-network.yaml file. Replace <name> with the name for this configuration.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: <name> 

   1

   
     namespace: openshift-sriov-network-operator 

   2

   
   spec:
     resourceName: <sriov_resource_name> 

   3

   
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true" 

   4

   
     priority: <priority> 

   5

   
     mtu: <mtu> 

   6

   
     numVfs: <num> 

   7

   
     nicSelector: 

   8

   
       vendor: "<vendor_code>" 

   9

   
       deviceID: "<device_id>" 

   10

   
       pfNames: ["<pf_name>", ...] 

   11

   
       rootDevices: ["<pci_bus_id>", "..."] 

   12

   
     deviceType: vfio-pci 

   13

   
     isRdma: false 

   14

   Copy to Clipboard Toggle word wrap

   1

   Specify a name for the CR object.

   2

   Specify the namespace where the SR-IOV Operator is installed.

   3

   Specify the resource name of the SR-IOV device plugin. You can create multiple SriovNetworkNodePolicy objects for a resource name.

   4

   Specify the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.

   5

   Optional: Specify an integer value between 0 and 99. A smaller number gets higher priority, so a priority of 10 is higher than a priority of 99. The default value is 99.

   6

   Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.

   7

   Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than 127.

   8

   The nicSelector mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters. It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally. If you specify rootDevices, you must also specify a value for vendor, deviceID, or pfNames. If you specify both pfNames and rootDevices at the same time, ensure that they point to an identical device.

   9

   Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either 8086 or 15b3.

   10

   Optional: Specify the device hex code of SR-IOV network device. The only allowed values are 158b, 1015, 1017.

   11

   Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.

   12

   The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: 0000:02:00.1.

   13

   The vfio-pci driver type is required for virtual functions in OpenShift Virtualization.

   14

   Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set isRdma to false. The default value is false.

   Note

   If isRDMA flag is set to true, you can continue to use the RDMA enabled VF as a normal network device. A device can be used in either mode.

2. Optional: Label the SR-IOV capable cluster nodes with SriovNetworkNodePolicy.Spec.NodeSelector if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".

3. Create the SriovNetworkNodePolicy object:

   $ oc create -f <name>-sriov-node-network.yaml

   Copy to Clipboard Toggle word wrap

   where <name> specifies the name for this configuration.

   After applying the configuration update, all the pods in sriov-network-operator namespace transition to the Running status.

4. To verify that the SR-IOV network device is configured, enter the following command. Replace <node_name> with the name of a node with the SR-IOV network device that you just configured.

   $ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'

   Copy to Clipboard Toggle word wrap

Procedure

1. Create the following SriovNetwork object, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

2. To create the object, enter the following command. Replace <name> with a name for this additional network.

   $ oc create -f <name>-sriov-network.yaml

   Copy to Clipboard Toggle word wrap

3. Optional: To confirm that the NetworkAttachmentDefinition object associated with the SriovNetwork object that you created in the previous step exists, enter the following command. Replace <namespace> with the namespace you specified in the SriovNetwork object.

   $ oc get net-attach-def -n <namespace>

   Copy to Clipboard Toggle word wrap

Procedure

1. Add the SR-IOV network details to the spec.domain.devices.interfaces and spec.networks stanzas of the VM configuration as in the following example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
   spec:
     domain:
       devices:
         interfaces:
         - name: nic1 

   1

   
           sriov: {}
     networks:
     - name: nic1 

   2

   
       multus:
           networkName: sriov-network 

   3

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specify a unique name for the SR-IOV interface.

   2

   Specify the name of the SR-IOV interface. This must be the same as the interfaces.name that you defined earlier.

   3

   Specify the name of the SR-IOV network attachment definition.

2. Apply the virtual machine configuration:

   $ oc apply -f <vm_sriov>.yaml 

   1

   Copy to Clipboard Toggle word wrap

   1

   The name of the virtual machine YAML file.

Procedure

1. Navigate to Virtualization VirtualMachines.
2. Click a VM to view the VirtualMachine details page.
3. On the Configuration tab, click the Network interfaces tab.
4. Click Add network interface.
5. Enter the interface name.
6. Select an SR-IOV network attachment definition from the Network list.
7. Select SR-IOV from the Type list.
8. Optional: Add a network Model or Mac address.
9. Click Save.
10. Restart or live-migrate the VM to apply the changes.

Procedure

1. Map your compute nodes topology to determine which Non-Uniform Memory Access (NUMA) CPUs are isolated for DPDK applications and which ones are reserved for the operating system (OS).

2. If your OpenShift Container Platform cluster uses separate control plane and compute nodes for high-availability:

   1. Label a subset of the compute nodes with a custom role; for example, worker-dpdk:

      $ oc label node <node_name> node-role.kubernetes.io/worker-dpdk=""

      Copy to Clipboard Toggle word wrap

   2. Create a new MachineConfigPool manifest that contains the worker-dpdk label in the spec.machineConfigSelector object:

      Example MachineConfigPool manifest

      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfigPool
      metadata:
        name: worker-dpdk
        labels:
          machineconfiguration.openshift.io/role: worker-dpdk
      spec:
        machineConfigSelector:
          matchExpressions:
            - key: machineconfiguration.openshift.io/role
              operator: In
              values:
                - worker
                - worker-dpdk
        nodeSelector:
          matchLabels:
            node-role.kubernetes.io/worker-dpdk: ""

      Copy to Clipboard Toggle word wrap

3. Create a PerformanceProfile manifest that applies to the labeled nodes and the machine config pool that you created in the previous steps. The performance profile specifies the CPUs that are isolated for DPDK applications and the CPUs that are reserved for house keeping.

   Example PerformanceProfile manifest

   apiVersion: performance.openshift.io/v2
   kind: PerformanceProfile
   metadata:
     name: profile-1
   spec:
     cpu:
       isolated: 4-39,44-79
       reserved: 0-3,40-43
     globallyDisableIrqLoadBalancing: true
     hugepages:
       defaultHugepagesSize: 1G
       pages:
       - count: 8
         node: 0
         size: 1G
     net:
       userLevelNetworking: true
     nodeSelector:
       node-role.kubernetes.io/worker-dpdk: ""
     numa:
       topologyPolicy: single-numa-node

   Copy to Clipboard Toggle word wrap

   Note

   The compute nodes automatically restart after you apply the MachineConfigPool and PerformanceProfile manifests.

4. Retrieve the name of the generated RuntimeClass resource from the status.runtimeClass field of the PerformanceProfile object:

   $ oc get performanceprofiles.performance.openshift.io profile-1 -o=jsonpath='{.status.runtimeClass}{"\n"}'

   Copy to Clipboard Toggle word wrap

5. Set the previously obtained RuntimeClass name as the default container runtime class for the virt-launcher pods by editing the HyperConverged custom resource (CR):

   $ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv \
       --type='json' -p='[{"op": "add", "path": "/spec/defaultRuntimeClass", "value":"<runtimeclass-name>"}]'

   Copy to Clipboard Toggle word wrap
   Note

   Editing the HyperConverged CR changes a global setting that affects all VMs that are created after the change is applied.

6. If your DPDK-enabled compute nodes use Simultaneous multithreading (SMT), enable the AlignCPUs enabler by editing the HyperConverged CR:

   $ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv \
       --type='json' -p='[{"op": "replace", "path": "/spec/featureGates/alignCPUs", "value": true}]'

   Copy to Clipboard Toggle word wrap
   Note

   Enabling AlignCPUs allows OpenShift Virtualization to request up to two additional dedicated CPUs to bring the total CPU count to an even parity when using emulator thread isolation.

7. Create an SriovNetworkNodePolicy object with the spec.deviceType field set to vfio-pci:

   Example SriovNetworkNodePolicy manifest

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: policy-1
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: intel_nics_dpdk
     deviceType: vfio-pci
     mtu: 9000
     numVfs: 4
     priority: 99
     nicSelector:
       vendor: "8086"
       deviceID: "1572"
       pfNames:
         - eno3
       rootDevices:
         - "0000:19:00.2"
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a namespace for your DPDK applications:

   $ oc create ns dpdk-ns

   Copy to Clipboard Toggle word wrap

2. Create an SriovNetwork object that references the SriovNetworkNodePolicy object. When you create an SriovNetwork object, the SR-IOV Network Operator automatically creates a NetworkAttachmentDefinition object.

   Example SriovNetwork manifest

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: dpdk-sriovnetwork
     namespace: openshift-sriov-network-operator
   spec:
     ipam: |
       {
         "type": "host-local",
         "subnet": "10.56.217.0/24",
         "rangeStart": "10.56.217.171",
         "rangeEnd": "10.56.217.181",
         "routes": [{
           "dst": "0.0.0.0/0"
         }],
         "gateway": "10.56.217.1"
       }
     networkNamespace: dpdk-ns 

   1

   
     resourceName: intel_nics_dpdk 

   2

   
     spoofChk: "off"
     trust: "on"
     vlan: 1019

   Copy to Clipboard Toggle word wrap

   1

   The namespace where the NetworkAttachmentDefinition object is deployed.

   2

   The value of the spec.resourceName attribute of the SriovNetworkNodePolicy object that was created when configuring the cluster for DPDK workloads.

3. Optional: Run the virtual machine latency checkup to verify that the network is properly configured.

Procedure

1. Edit the VirtualMachine manifest to include information about the SR-IOV network interface, CPU topology, CRI-O annotations, and huge pages:

   Example VirtualMachine manifest

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: rhel-dpdk-vm
   spec:
     runStrategy: Always
     template:
       metadata:
         annotations:
           cpu-load-balancing.crio.io: disable 

   1

   
           cpu-quota.crio.io: disable 

   2

   
           irq-load-balancing.crio.io: disable 

   3

   
       spec:
         domain:
           cpu:
             sockets: 1 

   4

   
             cores: 5 

   5

   
             threads: 2
             dedicatedCpuPlacement: true
             isolateEmulatorThread: true
           interfaces:
             - masquerade: {}
               name: default
             - model: virtio
               name: nic-east
               pciAddress: '0000:07:00.0'
               sriov: {}
             networkInterfaceMultiqueue: true
             rng: {}
         memory:
           hugepages:
             pageSize: 1Gi 

   6

   
             guest: 8Gi
         networks:
           - name: default
             pod: {}
           - multus:
               networkName: dpdk-net 

   7

   
             name: nic-east
   # ...

   Copy to Clipboard Toggle word wrap

   1

   This annotation specifies that load balancing is disabled for CPUs that are used by the container.

   2

   This annotation specifies that the CPU quota is disabled for CPUs that are used by the container.

   3

   This annotation specifies that Interrupt Request (IRQ) load balancing is disabled for CPUs that are used by the container.

   4

   The number of sockets inside the VM. This field must be set to 1 for the CPUs to be scheduled from the same Non-Uniform Memory Access (NUMA) node.

   5

   The number of cores inside the VM. This must be a value greater than or equal to 1. In this example, the VM is scheduled with 5 hyper-threads or 10 CPUs.

   6

   The size of the huge pages. The possible values for x86-64 architecture are 1Gi and 2Mi. In this example, the request is for 8 huge pages of size 1Gi.

   7

   The name of the SR-IOV NetworkAttachmentDefinition object.

2. Save and exit the editor.

3. Apply the VirtualMachine manifest:

   $ oc apply -f <file_name>.yaml

   Copy to Clipboard Toggle word wrap

4. Configure the guest operating system. The following example shows the configuration steps for RHEL 9 operating system:

   1. Configure huge pages by using the GRUB bootloader command-line interface. In the following example, 8 1G huge pages are specified.

      $ grubby --update-kernel=ALL --args="default_hugepagesz=1GB hugepagesz=1G hugepages=8"

      Copy to Clipboard Toggle word wrap

   2. To achieve low-latency tuning by using the cpu-partitioning profile in the TuneD application, run the following commands:

      $ dnf install -y tuned-profiles-cpu-partitioning

      Copy to Clipboard Toggle word wrap

      $ echo isolated_cores=2-9 > /etc/tuned/cpu-partitioning-variables.conf

      Copy to Clipboard Toggle word wrap

      The first two CPUs (0 and 1) are set aside for house keeping tasks and the rest are isolated for the DPDK application.

      $ tuned-adm profile cpu-partitioning

      Copy to Clipboard Toggle word wrap

   3. Override the SR-IOV NIC driver by using the driverctl device driver control utility:

      $ dnf install -y driverctl

      Copy to Clipboard Toggle word wrap

      $ driverctl set-override 0000:07:00.0 vfio-pci

      Copy to Clipboard Toggle word wrap
5. Restart the VM to apply the changes.

Procedure

1. Use your preferred text editor to edit the VirtualMachine manifest, as shown in the following example:

   Example VM configuration

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: vm-fedora
   template:
     spec:
       domain:
         devices:
           interfaces:
           - name: defaultnetwork
             masquerade: {}
           # new interface
           - name: <secondary_nic> 

   1

   
             bridge: {}
       networks:
       - name: defaultnetwork
         pod: {}
       # new network
       - name: <secondary_nic> 

   2

   
         multus:
           networkName: <nad_name> 

   3

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specifies the name of the new network interface.

   2

   Specifies the name of the network. This must be the same as the name of the new network interface that you defined in the template.spec.domain.devices.interfaces list.

   3

   Specifies the name of the NetworkAttachmentDefinition object.

2. Save your changes and exit the editor.

3. For the new configuration to take effect, apply the changes by running the following command. Applying the changes triggers automatic VM live migration and attaches the network interface to the running VM.

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your VirtualMachine manifest YAML file.

Verification

1. Verify that the VM live migration is successful by using the following command:

   $ oc get VirtualMachineInstanceMigration -w

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                        PHASE             VMI
   kubevirt-migrate-vm-lj62q   Scheduling        vm-fedora
   kubevirt-migrate-vm-lj62q   Scheduled         vm-fedora
   kubevirt-migrate-vm-lj62q   PreparingTarget   vm-fedora
   kubevirt-migrate-vm-lj62q   TargetReady       vm-fedora
   kubevirt-migrate-vm-lj62q   Running           vm-fedora
   kubevirt-migrate-vm-lj62q   Succeeded         vm-fedora

   Copy to Clipboard Toggle word wrap

2. Verify that the new interface is added to the VM by checking the status of the virtual machine instance (VMI):

   $ oc get vmi vm-fedora -ojsonpath="{ @.status.interfaces }"

   Copy to Clipboard Toggle word wrap

   Example output

   [
     {
       "infoSource": "domain, guest-agent",
       "interfaceName": "eth0",
       "ipAddress": "10.130.0.195",
       "ipAddresses": [
         "10.130.0.195",
         "fd02:0:0:3::43c"
       ],
       "mac": "52:54:00:0e:ab:25",
       "name": "default",
       "queueCount": 1
     },
     {
       "infoSource": "domain, guest-agent, multus-status",
       "interfaceName": "eth1",
       "mac": "02:d8:b8:00:00:2a",
       "name": "bridge-interface", 

   1

   
       "queueCount": 1
     }
   ]

   Copy to Clipboard Toggle word wrap

   1

   The hot plugged interface appears in the VMI status.

Procedure

1. Using your preferred text editor, edit the VirtualMachine manifest file and set the interface state to absent. Setting the interface state to absent detaches the network interface from the guest, but the interface still exists in the pod.

   Example VM configuration

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: vm-fedora
   template:
     spec:
       domain:
         devices:
           interfaces:
             - name: defaultnetwork
               masquerade: {}
             # set the interface state to absent
             - name: <secondary_nic>
               state: absent 

   1

   
               bridge: {}
       networks:
         - name: defaultnetwork
           pod: {}
         - name: <secondary_nic>
           multus:
             networkName: <nad_name>
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Set the interface state to absent to detach it from the running VM. Removing the interface details from the VM specification does not hot unplug the secondary network interface.

2. Save your changes and exit the editor.

3. For the new configuration to take effect, apply the changes by running the following command. Applying the changes triggers automatic VM live migration and removes the interface from the pod.

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap

   where:

   <filename>

   Specifies the name of your VirtualMachine manifest YAML file.

Procedure

1. Navigate to Virtualization VirtualMachines.
2. Select a VM to view the VirtualMachine details page.
3. On the Configuration tab, click Network. A list of network interfaces is displayed.
4. Click the Options menu of the interface that you want to edit.

5. Choose the appropriate option to set the interface link state:

   • If the current interface link state is up, select Set link down.
   • If the current interface link state is down, select Set link up.

Procedure

1. Edit the VM configuration to set the interface link state, as in the following example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: my-vm
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: default 

   1

   
                 state: down 

   2

   
                 masquerade: { }
         networks:
           - name: default
             pod: { }
   # ...

   Copy to Clipboard Toggle word wrap

   1

   The name of the interface.

   2

   The state of the interface. The possible values are:

   • up: Represents an active network connection. This is the default if no value is specified.
   • down: Represents a network interface link that is switched off.

   • absent: Represents a network interface that is hot unplugged.

     Important

     If you have defined readiness or liveness probes to run VM health checks, setting the primary interface’s link state to down causes the probes to fail. If a liveness probe fails, the VM is deleted and a new VM is created to restore responsiveness.

2. Apply the VirtualMachine manifest:

   $ oc apply -f <filename>.yaml

   Copy to Clipboard Toggle word wrap