Chapter 8. Networking

Procedure

1. Edit the interfaces spec of your virtual machine configuration file:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: default
                 masquerade: {} 

   1

   
                 ports: 

   2

   
                   - port: 80
   # ...
         networks:
         - name: default
           pod: {}

   Copy to Clipboard Toggle word wrap

   1

   Connect using masquerade mode.

   2

   Optional: List the ports that you want to expose from the virtual machine, each specified by the port field. The port value must be a number between 0 and 65536. When the ports array is not used, all ports in the valid range are open to incoming traffic. In this example, incoming traffic is allowed on port 80.

   Note

   Ports 49152 and 49153 are reserved for use by the libvirt platform and all other incoming traffic to these ports is dropped.

2. Create the virtual machine:

   $ oc create -f <vm-name>.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. In a new virtual machine configuration, include an interface with masquerade and configure the IPv6 address and default gateway by using cloud-init.

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm-ipv6
   spec:
     template:
       spec:
         domain:
           devices:
             interfaces:
               - name: default
                 masquerade: {} 

   1

   
                 ports:
                   - port: 80 

   2

   
   # ...
         networks:
         - name: default
           pod: {}
         volumes:
         - cloudInitNoCloud:
             networkData: |
               version: 2
               ethernets:
                 eth0:
                   dhcp4: true
                   addresses: [ fd10:0:2::2/120 ] 

   3

   
                   gateway6: fd10:0:2::1 

   4

   Copy to Clipboard Toggle word wrap

   1

   Connect using masquerade mode.

   2

   Allows incoming traffic on port 80 to the virtual machine.

   3

   The static IPv6 address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::2/120.

   4

   The gateway IP address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::1.

2. Create the virtual machine in the namespace:

   $ oc create -f example-vm-ipv6.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Edit the VirtualMachine manifest to add the label for service creation:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
     namespace: example-namespace
   spec:
     running: false
     template:
       metadata:
         labels:
           special: key 

   1

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Add special: key to the spec.template.metadata.labels stanza.

   Note

   Labels on a virtual machine are passed through to the pod. The special: key label must match the label in the spec.selector attribute of the Service manifest.

2. Save the VirtualMachine manifest file to apply your changes.

3. Create a Service manifest to expose the VM:

   apiVersion: v1
   kind: Service
   metadata:
     name: example-service
     namespace: example-namespace
   spec:
   # ...
     selector:
       special: key 

   1

   
     type: NodePort 

   2

   
     ports: 

   3

   
       protocol: TCP
       port: 80
       targetPort: 9376
       nodePort: 30000

   Copy to Clipboard Toggle word wrap

   1

   Specify the label that you added to the spec.template.metadata.labels stanza of the VirtualMachine manifest.

   2

   Specify ClusterIP, NodePort, or LoadBalancer.

   3

   Specifies a collection of network ports and protocols that you want to expose from the virtual machine.

4. Save the Service manifest file.

5. Create the service by running the following command:

   $ oc create -f example-service.yaml

   Copy to Clipboard Toggle word wrap
6. Restart the VM to apply the changes.

Procedure

1. Create an SriovNetworkNodePolicy object, and then save the YAML in the <name>-sriov-node-network.yaml file. Replace <name> with the name for this configuration.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: <name> 

   1

   
     namespace: openshift-sriov-network-operator 

   2

   
   spec:
     resourceName: <sriov_resource_name> 

   3

   
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true" 

   4

   
     priority: <priority> 

   5

   
     mtu: <mtu> 

   6

   
     numVfs: <num> 

   7

   
     nicSelector: 

   8

   
       vendor: "<vendor_code>" 

   9

   
       deviceID: "<device_id>" 

   10

   
       pfNames: ["<pf_name>", ...] 

   11

   
       rootDevices: ["<pci_bus_id>", "..."] 

   12

   
     deviceType: vfio-pci 

   13

   
     isRdma: false 

   14

   Copy to Clipboard Toggle word wrap

   1

   Specify a name for the CR object.

   2

   Specify the namespace where the SR-IOV Operator is installed.

   3

   Specify the resource name of the SR-IOV device plugin. You can create multiple SriovNetworkNodePolicy objects for a resource name.

   4

   Specify the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.

   5

   Optional: Specify an integer value between 0 and 99. A smaller number gets higher priority, so a priority of 10 is higher than a priority of 99. The default value is 99.

   6

   Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.

   7

   Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than 127.

   8

   The nicSelector mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters. It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally. If you specify rootDevices, you must also specify a value for vendor, deviceID, or pfNames. If you specify both pfNames and rootDevices at the same time, ensure that they point to an identical device.

   9

   Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either 8086 or 15b3.

   10

   Optional: Specify the device hex code of SR-IOV network device. The only allowed values are 158b, 1015, 1017.

   11

   Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.

   12

   The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: 0000:02:00.1.

   13

   The vfio-pci driver type is required for virtual functions in OpenShift Virtualization.

   14

   Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set isRdma to false. The default value is false.

   Note

   If isRDMA flag is set to true, you can continue to use the RDMA enabled VF as a normal network device. A device can be used in either mode.

2. Optional: Label the SR-IOV capable cluster nodes with SriovNetworkNodePolicy.Spec.NodeSelector if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".

3. Create the SriovNetworkNodePolicy object:

   $ oc create -f <name>-sriov-node-network.yaml

   Copy to Clipboard Toggle word wrap

   where <name> specifies the name for this configuration.

   After applying the configuration update, all the pods in sriov-network-operator namespace transition to the Running status.

4. To verify that the SR-IOV network device is configured, enter the following command. Replace <node_name> with the name of a node with the SR-IOV network device that you just configured.

   $ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'

   Copy to Clipboard Toggle word wrap

Procedure

1. Create the following SriovNetwork object, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

2. To create the object, enter the following command. Replace <name> with a name for this additional network.

   $ oc create -f <name>-sriov-network.yaml

   Copy to Clipboard Toggle word wrap

3. Optional: To confirm that the NetworkAttachmentDefinition object associated with the SriovNetwork object that you created in the previous step exists, enter the following command. Replace <namespace> with the namespace you specified in the SriovNetwork object.

   $ oc get net-attach-def -n <namespace>

   Copy to Clipboard Toggle word wrap

Procedure

1. Add the SR-IOV network details to the spec.domain.devices.interfaces and spec.networks stanzas of the VM configuration as in the following example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
   spec:
     domain:
       devices:
         interfaces:
         - name: default
           masquerade: {}
         - name: nic1 

   1

   
           sriov: {}
     networks:
     - name: default
       pod: {}
     - name: nic1 

   2

   
       multus:
           networkName: sriov-network 

   3

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specify a unique name for the SR-IOV interface.

   2

   Specify the name of the SR-IOV interface. This must be the same as the interfaces.name that you defined earlier.

   3

   Specify the name of the SR-IOV network attachment definition.

2. Apply the virtual machine configuration:

   $ oc apply -f <vm_sriov>.yaml 

   1

   Copy to Clipboard Toggle word wrap

   1

   The name of the virtual machine YAML file.

Procedure

1. Map your compute nodes topology to determine which Non-Uniform Memory Access (NUMA) CPUs are isolated for DPDK applications and which ones are reserved for the operating system (OS).

2. Label a subset of the compute nodes with a custom role; for example, worker-dpdk:

   $ oc label node <node_name> node-role.kubernetes.io/worker-dpdk=""

   Copy to Clipboard Toggle word wrap

3. Create a new MachineConfigPool manifest that contains the worker-dpdk label in the spec.machineConfigSelector object:

   Example MachineConfigPool manifest

   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfigPool
   metadata:
     name: worker-dpdk
     labels:
       machineconfiguration.openshift.io/role: worker-dpdk
   spec:
     machineConfigSelector:
       matchExpressions:
         - key: machineconfiguration.openshift.io/role
           operator: In
           values:
             - worker
             - worker-dpdk
     nodeSelector:
       matchLabels:
         node-role.kubernetes.io/worker-dpdk: ""

   Copy to Clipboard Toggle word wrap

4. Create a PerformanceProfile manifest that applies to the labeled nodes and the machine config pool that you created in the previous steps. The performance profile specifies the CPUs that are isolated for DPDK applications and the CPUs that are reserved for house keeping.

   Example PerformanceProfile manifest

   apiVersion: performance.openshift.io/v2
   kind: PerformanceProfile
   metadata:
     name: profile-1
   spec:
     cpu:
       isolated: 4-39,44-79
       reserved: 0-3,40-43
     globallyDisableIrqLoadBalancing: true
     hugepages:
       defaultHugepagesSize: 1G
       pages:
       - count: 8
         node: 0
         size: 1G
     net:
       userLevelNetworking: true
     nodeSelector:
       node-role.kubernetes.io/worker-dpdk: ""
     numa:
       topologyPolicy: single-numa-node

   Copy to Clipboard Toggle word wrap

   Note

   The compute nodes automatically restart after you apply the MachineConfigPool and PerformanceProfile manifests.

5. Retrieve the name of the generated RuntimeClass resource from the status.runtimeClass field of the PerformanceProfile object:

   $ oc get performanceprofiles.performance.openshift.io profile-1 -o=jsonpath='{.status.runtimeClass}{"\n"}'

   Copy to Clipboard Toggle word wrap

6. Set the previously obtained RuntimeClass name as the default container runtime class for the virt-launcher pods by editing the HyperConverged custom resource (CR):

   $ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv \
       --type='json' -p='[{"op": "add", "path": "/spec/defaultRuntimeClass", "value":"<runtimeclass-name>"}]'

   Copy to Clipboard Toggle word wrap
   Note

   Editing the HyperConverged CR changes a global setting that affects all VMs that are created after the change is applied.

7. Create an SriovNetworkNodePolicy object with the spec.deviceType field set to vfio-pci:

   Example SriovNetworkNodePolicy manifest

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: policy-1
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: intel_nics_dpdk
     deviceType: vfio-pci
     mtu: 9000
     numVfs: 4
     priority: 99
     nicSelector:
       vendor: "8086"
       deviceID: "1572"
       pfNames:
         - eno3
       rootDevices:
         - "0000:19:00.2"
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a namespace for your DPDK applications:

   $ oc create ns dpdk-checkup-ns

   Copy to Clipboard Toggle word wrap

2. Create an SriovNetwork object that references the SriovNetworkNodePolicy object. When you create an SriovNetwork object, the SR-IOV Network Operator automatically creates a NetworkAttachmentDefinition object.

   Example SriovNetwork manifest

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: dpdk-sriovnetwork
     namespace: openshift-sriov-network-operator
   spec:
     ipam: |
       {
         "type": "host-local",
         "subnet": "10.56.217.0/24",
         "rangeStart": "10.56.217.171",
         "rangeEnd": "10.56.217.181",
         "routes": [{
           "dst": "0.0.0.0/0"
         }],
         "gateway": "10.56.217.1"
       }
     networkNamespace: dpdk-checkup-ns 

   1

   
     resourceName: intel_nics_dpdk 

   2

   
     spoofChk: "off"
     trust: "on"
     vlan: 1019

   Copy to Clipboard Toggle word wrap

   1

   The namespace where the NetworkAttachmentDefinition object is deployed.

   2

   The value of the spec.resourceName attribute of the SriovNetworkNodePolicy object that was created when configuring the cluster for DPDK workloads.

3. Optional: Run the virtual machine latency checkup to verify that the network is properly configured.
4. Optional: Run the DPDK checkup to verify that the namespace is ready for DPDK workloads.

Procedure

1. Edit the VirtualMachine manifest to include information about the SR-IOV network interface, CPU topology, CRI-O annotations, and huge pages:

   Example VirtualMachine manifest

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: rhel-dpdk-vm
   spec:
     running: true
     template:
       metadata:
         annotations:
           cpu-load-balancing.crio.io: disable 

   1

   
           cpu-quota.crio.io: disable 

   2

   
           irq-load-balancing.crio.io: disable 

   3

   
       spec:
         domain:
           cpu:
             sockets: 1 

   4

   
             cores: 5 

   5

   
             threads: 2
             dedicatedCpuPlacement: true
             isolateEmulatorThread: true
           interfaces:
             - masquerade: {}
               name: default
             - model: virtio
               name: nic-east
               pciAddress: '0000:07:00.0'
               sriov: {}
             networkInterfaceMultiqueue: true
             rng: {}
         memory:
           hugepages:
             pageSize: 1Gi 

   6

   
             guest: 8Gi
         networks:
           - name: default
             pod: {}
           - multus:
               networkName: dpdk-net 

   7

   
             name: nic-east
   # ...

   Copy to Clipboard Toggle word wrap

   1

   This annotation specifies that load balancing is disabled for CPUs that are used by the container.

   2

   This annotation specifies that the CPU quota is disabled for CPUs that are used by the container.

   3

   This annotation specifies that Interrupt Request (IRQ) load balancing is disabled for CPUs that are used by the container.

   4

   The number of sockets inside the VM. This field must be set to 1 for the CPUs to be scheduled from the same Non-Uniform Memory Access (NUMA) node.

   5

   The number of cores inside the VM. This must be a value greater than or equal to 1. In this example, the VM is scheduled with 5 hyper-threads or 10 CPUs.

   6

   The size of the huge pages. The possible values for x86-64 architecture are 1Gi and 2Mi. In this example, the request is for 8 huge pages of size 1Gi.

   7

   The name of the SR-IOV NetworkAttachmentDefinition object.

2. Save and exit the editor.

3. Apply the VirtualMachine manifest:

   $ oc apply -f <file_name>.yaml

   Copy to Clipboard Toggle word wrap

4. Configure the guest operating system. The following example shows the configuration steps for RHEL 8 OS:

   1. Configure huge pages by using the GRUB bootloader command-line interface. In the following example, 8 1G huge pages are specified.

      $ grubby --update-kernel=ALL --args="default_hugepagesz=1GB hugepagesz=1G hugepages=8"

      Copy to Clipboard Toggle word wrap

   2. To achieve low-latency tuning by using the cpu-partitioning profile in the TuneD application, run the following commands:

      $ dnf install -y tuned-profiles-cpu-partitioning

      Copy to Clipboard Toggle word wrap

      $ echo isolated_cores=2-9 > /etc/tuned/cpu-partitioning-variables.conf

      Copy to Clipboard Toggle word wrap

      The first two CPUs (0 and 1) are set aside for house keeping tasks and the rest are isolated for the DPDK application.

      $ tuned-adm profile cpu-partitioning

      Copy to Clipboard Toggle word wrap

   3. Override the SR-IOV NIC driver by using the driverctl device driver control utility:

      $ dnf install -y driverctl

      Copy to Clipboard Toggle word wrap

      $ driverctl set-override 0000:07:00.0 vfio-pci

      Copy to Clipboard Toggle word wrap
5. Restart the VM to apply the changes.

Procedure

1. If the VM to which you want to hot plug the network interface is not running, start it by using the following command:

   $ virtctl start <vm_name> -n <namespace>

   Copy to Clipboard Toggle word wrap

2. Use the following command to hot plug a new network interface to the running VM. The virtctl addinterface command adds the new network interface to the VM and virtual machine instance (VMI) specification but does not attach it to the running VM.

   $ virtctl addinterface <vm_name> --network-attachment-definition-name <net_attach_dev_namespace>/<net_attach_def_name> --name <interface_name>

   Copy to Clipboard Toggle word wrap

   where:

   <vm_name>

   The name of the VirtualMachine object.

   <net_attach_def_name>

   The name of the NetworkAttachmentDefinition object.

   <net_attach_dev_namespace>

   An identifier for the namespace associated with the NetworkAttachmentDefinition object. The supported values are default or the name of the namespace where the VM is located.

   <interface_name>

   The name of the new network interface.

3. To attach the network interface to the running VM, live migrate the VM by using the following command:

   $ virtctl migrate <vm_name>

   Copy to Clipboard Toggle word wrap

Verification

1. Verify that the VM live migration is successful by using the following command:

   $ oc get VirtualMachineInstanceMigration -w

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                        PHASE             VMI
   kubevirt-migrate-vm-lj62q   Scheduling        vm-fedora
   kubevirt-migrate-vm-lj62q   Scheduled         vm-fedora
   kubevirt-migrate-vm-lj62q   PreparingTarget   vm-fedora
   kubevirt-migrate-vm-lj62q   TargetReady       vm-fedora
   kubevirt-migrate-vm-lj62q   Running           vm-fedora
   kubevirt-migrate-vm-lj62q   Succeeded         vm-fedora

   Copy to Clipboard Toggle word wrap

2. Verify that the new interface is added to the VM by checking the VMI status:

   $ oc get vmi vm-fedora -ojsonpath="{ @.status.interfaces }"

   Copy to Clipboard Toggle word wrap

   Example output

   [
     {
       "infoSource": "domain, guest-agent",
       "interfaceName": "eth0",
       "ipAddress": "10.130.0.195",
       "ipAddresses": [
         "10.130.0.195",
         "fd02:0:0:3::43c"
       ],
       "mac": "52:54:00:0e:ab:25",
       "name": "default",
       "queueCount": 1
     },
     {
       "infoSource": "domain, guest-agent, multus-status",
       "interfaceName": "eth1",
       "mac": "02:d8:b8:00:00:2a",
       "name": "bridge-interface", 

   1

   
       "queueCount": 1
     }
   ]

   Copy to Clipboard Toggle word wrap

   1

   The hot plugged interface appears in the VMI status.

Procedure

1. Hot unplug a bridge network interface by running the following command. The virtctl removeinterface command detaches the network interface from the guest, but the interface still exists in the pod.

   $ virtctl removeinterface <vm_name> --name <interface_name>

   Copy to Clipboard Toggle word wrap

2. Remove the interface from the pod by migrating the VM:

   $ virtctl migrate <vm_name>

   Copy to Clipboard Toggle word wrap

Procedure

1. Edit the VM configuration file to add the sidecar.istio.io/inject: "true" annotation:

   Example configuration file

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     labels:
       kubevirt.io/vm: vm-istio
     name: vm-istio
   spec:
     runStrategy: Always
     template:
       metadata:
         labels:
           kubevirt.io/vm: vm-istio
           app: vm-istio 

   1

   
         annotations:
           sidecar.istio.io/inject: "true" 

   2

   
       spec:
         domain:
           devices:
             interfaces:
             - name: default
               masquerade: {} 

   3

   
             disks:
             - disk:
                 bus: virtio
               name: containerdisk
             - disk:
                 bus: virtio
               name: cloudinitdisk
           resources:
             requests:
               memory: 1024M
         networks:
         - name: default
           pod: {}
         terminationGracePeriodSeconds: 180
         volumes:
         - containerDisk:
             image: registry:5000/kubevirt/fedora-cloud-container-disk-demo:devel
           name: containerdisk

   Copy to Clipboard Toggle word wrap

   1

   The key/value pair (label) that must be matched to the service selector attribute.

   2

   The annotation to enable automatic sidecar injection.

   3

   The binding method (masquerade mode) for use with the default pod network.

2. Apply the VM configuration:

   $ oc apply -f <vm_name>.yaml 

   1

   Copy to Clipboard Toggle word wrap

   1

   The name of the virtual machine YAML file.

3. Create a Service object to expose your VM to the service mesh.

   apiVersion: v1
   kind: Service
   metadata:
     name: vm-istio
   spec:
     selector:
       app: vm-istio 

   1

   
     ports:
       - port: 8080
         name: http
         protocol: TCP

   Copy to Clipboard Toggle word wrap

   1

   The service selector that determines the set of pods targeted by a service. This attribute corresponds to the spec.metadata.labels field in the VM configuration file. In the above example, the Service object named vm-istio targets TCP port 8080 on any pod with the label app=vm-istio.

4. Create the service:

   $ oc create -f <service_name>.yaml 

   1

   Copy to Clipboard Toggle word wrap

   1

   The name of the service YAML file.

Procedure

1. Create a NetworkAttachmentDefinition manifest according to the following example:

   Example configuration file

   apiVersion: "k8s.cni.cncf.io/v1"
   kind: NetworkAttachmentDefinition
   metadata:
     name: my-secondary-network 

   1

   
     namespace: openshift-cnv 

   2

   
   spec:
     config: '{
       "cniVersion": "0.3.1",
       "name": "migration-bridge",
       "type": "macvlan",
       "master": "eth1", 

   3

   
       "mode": "bridge",
       "ipam": {
         "type": "whereabouts", 

   4

   
         "range": "10.200.5.0/24" 

   5

   
       }
     }'

   Copy to Clipboard Toggle word wrap

   1

   Specify the name of the NetworkAttachmentDefinition object.

   2 3

   Specify the name of the NIC to be used for live migration.

   4

   Specify the name of the CNI plugin that provides the network for the NAD.

   5

   Specify an IP address range for the secondary network. This range must not overlap the IP addresses of the main network.

2. Open the HyperConverged CR in your default editor by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

   Copy to Clipboard Toggle word wrap

3. Add the name of the NetworkAttachmentDefinition object to the spec.liveMigrationConfig stanza of the HyperConverged CR:

   Example HyperConverged manifest

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     liveMigrationConfig:
       completionTimeoutPerGiB: 800
       network: <network> 

   1

   
       parallelMigrationsPerCluster: 5
       parallelOutboundMigrationsPerNode: 2
       progressTimeout: 150
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specify the name of the Multus NetworkAttachmentDefinition object to be used for live migrations.

4. Save your changes and exit the editor. The virt-handler pods restart and connect to the secondary network.

Procedure

1. Navigate to Virtualization > Overview in the OpenShift Container Platform web console.
2. Click the Settings tab and then click Live migration.
3. Select the network from the Live migration network list.

Procedure

1. Create a load balancer service to expose the DNS server outside the cluster by running the oc expose command according to the following example:

   $ oc expose -n openshift-cnv deployment/secondary-dns --name=dns-lb \
     --type=LoadBalancer --port=53 --target-port=5353 --protocol='UDP'

   Copy to Clipboard Toggle word wrap

2. Retrieve the external IP address by running the following command:

   $ oc get service -n openshift-cnv

   Copy to Clipboard Toggle word wrap

   Example output

   NAME       TYPE             CLUSTER-IP     EXTERNAL-IP      PORT(S)          AGE
   dns-lb     LoadBalancer     172.30.27.5    10.46.41.94      53:31829/TCP     5s

   Copy to Clipboard Toggle word wrap

3. Edit the HyperConverged CR in your default editor by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

   Copy to Clipboard Toggle word wrap

4. Enable the DNS server and monitoring components according to the following example:

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     featureGates:
       deployKubeSecondaryDNS: true
     kubeSecondaryDNSNameServerIP: "10.46.41.94" 

   1

   
   # ...

   Copy to Clipboard Toggle word wrap

   1

   Specify the external IP address exposed by the load balancer service.

5. Save the file and exit the editor.

6. Retrieve the cluster FQDN by running the following command:

    $ oc get dnses.config.openshift.io cluster -o jsonpath='{.spec.baseDomain}'

   Copy to Clipboard Toggle word wrap

   Example output

   openshift.example.com

   Copy to Clipboard Toggle word wrap

7. Point to the DNS server by using one of the following methods:

   • Add the kubeSecondaryDNSNameServerIP value to the resolv.conf file on your local machine.

     Note

     Editing the resolv.conf file overwrites existing DNS settings.

   • Add the kubeSecondaryDNSNameServerIP value and the cluster FQDN to the enterprise DNS server records. For example:

     vm.<FQDN>. IN NS ns.vm.<FQDN>.

     Copy to Clipboard Toggle word wrap

     ns.vm.<FQDN>. IN A 10.46.41.94

     Copy to Clipboard Toggle word wrap

Procedure

1. Retrieve the network interface name from the VM configuration by running the following command:

   $ oc get vm -n <namespace> <vm_name> -o yaml

   Copy to Clipboard Toggle word wrap

   Example output

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm
     namespace: example-namespace
   spec:
     running: true
     template:
       spec:
         domain:
           devices:
             interfaces:
               - bridge: {}
                 name: example-nic
   # ...
         networks:
         - multus:
             networkName: bridge-conf
           name: example-nic 

   1

   Copy to Clipboard Toggle word wrap

   1

   Note the name of the network interface.

2. Connect to the VM by using the ssh command:

   $ ssh <user_name>@<interface_name>.<vm_name>.<namespace>.vm.<cluster_fqdn>

   Copy to Clipboard Toggle word wrap