Chapter 10. Virtual machines

Procedure

1. Click Virtualization VirtualMachines from the side menu.
2. Click Create and select With YAML.

3. Write or paste your virtual machine configuration in the editable window.

   1. Alternatively, use the example virtual machine provided by default in the YAML screen.
4. Optional: Click Download to download the YAML configuration file in its present state.
5. Click Create to create the virtual machine.

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Select a virtual machine to open the VirtualMachine details page.
3. Click the Console tab. The VNC console opens by default.
4. Click Disconnect to ensure that only one console session is open at a time. Otherwise, the VNC console session remains active in the background.
5. Click the VNC Console drop-down list and select Serial Console.
6. Click Disconnect to end the console session.
7. Optional: Open the serial console in a separate window by clicking Open Console in New Window.

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Select a virtual machine to open the VirtualMachine details page.
3. Click the Console tab. The VNC console opens by default.
4. Optional: Open the VNC console in a separate window by clicking Open Console in New Window.
5. Optional: Send key combinations to the virtual machine by clicking Send Key.
6. Click outside the console window and then click Disconnect to end the session.

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Click a Windows virtual machine to open the VirtualMachine details page.
3. Click the Console tab.
4. From the list of consoles, select Desktop viewer.
5. Click Launch Remote Desktop to download the console.rdp file.
6. Reference the console.rdp file in your preferred RDP client to connect to the Windows virtual machine.

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines
2. Select a Windows virtual machine to open the Overview screen.
3. Click the Console tab.
4. From the list of consoles, select VNC console.

5. Choose the appropriate key combination from the Send Key list:

   1. To access the default VM display, select Ctl + Alt+ 1.
   2. To access the vGPU display, select Ctl + Alt + 2.

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Click the Options menu for your virtual machine and select Copy SSH command.
3. Paste it in the terminal to access the VM.

Procedure

1. Configure SSH key authentication:

   1. Use the ssh-keygen command to generate an SSH public key pair:

      $ ssh-keygen -f <key_file> 1

      1

      Specify the file in which to store the keys.

   2. Create an SSH authentication secret which contains the SSH public key to access the VM:

      $ oc create secret generic my-pub-key --from-file=key1=<key_file>.pub

   3. Add a reference to the secret in the VirtualMachine manifest. For example:

      apiVersion: kubevirt.io/v1
      kind: VirtualMachine
      metadata:
        name: testvm
      spec:
        running: true
        template:
          spec:
            accessCredentials:
            - sshPublicKey:
                source:
                  secret:
                    secretName: my-pub-key 1
                propagationMethod:
                  configDrive: {} 2
      # ...

      1

      Reference to the SSH authentication Secret object.

      2

      The SSH public key is injected into the VM as cloud-init metadata using the configDrive provider.

   4. Restart the VM to apply your changes.

2. Connect to the VM via SSH:

   1. Run the following command to access the VM via SSH:

      $ virtctl ssh -i <key_file> <vm_username>@<vm_name>

   2. Optional: To securely transfer files to or from the VM, use the following commands:

      Copy a file from your machine to the VM

      $ virtctl scp -i <key_file> <filename> <vm_username>@<vm_name>:

      Copy a file from the VM to your machine

      $ virtctl scp -i <key_file> <vm_username@<vm_name>:<filename> .

Procedure

1. Add the following text to the ~/.ssh/config file on your client machine:

   Host vm/*
     ProxyCommand virtctl port-forward --stdio=true %h %p

2. Connect to the VM by running the following command:

   $ ssh <user>@vm/<vm_name>.<namespace>

Procedure

1. Connect to the graphical interface with the virtctl utility:

   $ virtctl vnc <VMI>

2. If the command failed, try using the -v flag to collect troubleshooting information:

   $ virtctl vnc <VMI> -v 4

Procedure

1. Edit the VirtualMachine manifest to add the label for service creation:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: vm-ephemeral
     namespace: example-namespace
   spec:
     running: false
     template:
       metadata:
         labels:
           special: key 1
   # ...

   1

   Add the label special: key in the spec.template.metadata.labels section.

   Note

   Labels on a virtual machine are passed through to the pod. The special: key label must match the label in the spec.selector attribute of the Service manifest.

2. Save the VirtualMachine manifest file to apply your changes.

3. Create a Service manifest to expose the VM:

   apiVersion: v1
   kind: Service
   metadata:
     name: rdpservice 1
     namespace: example-namespace 2
   spec:
     ports:
     - targetPort: 3389 3
       protocol: TCP
     selector:
       special: key 4
     type: NodePort 5
   # ...

   1

   The name of the Service object.

   2

   The namespace where the Service object resides. This must match the metadata.namespace field of the VirtualMachine manifest.

   3

   The VM port to be exposed by the service. It must reference an open port if a port list is defined in the VM manifest.

   4

   The reference to the label that you added in the spec.template.metadata.labels stanza of the VirtualMachine manifest.

   5

   The type of service.

4. Save the Service manifest file.

5. Create the service by running the following command:

   $ oc create -f <service_name>.yaml

6. Start the VM. If the VM is already running, restart it.

7. Query the Service object to verify that it is available:

   $ oc get service -n example-namespace

   Example output for NodePort service

   NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)            AGE
   rdpservice   NodePort    172.30.232.73   <none>       3389:30000/TCP    5m

8. Run the following command to obtain the IP address for the node:

   $ oc get node <node_name> -o wide

   Example output

   NAME    STATUS   ROLES   AGE    VERSION  INTERNAL-IP      EXTERNAL-IP
   node01  Ready    worker  6d22h  v1.24.0  192.168.55.101   <none>

9. Specify the node IP address and the assigned port in your preferred RDP client.
10. Enter the user name and password to connect to the Windows virtual machine.

Procedure

1. Access the virtual machine command line through one of the consoles or by SSH.

2. Install the QEMU guest agent on the virtual machine:

   $ yum install -y qemu-guest-agent

3. Ensure the service is persistent and start it:

   $ systemctl enable --now qemu-guest-agent

Verification

1. Run the following command to verify that AgentConnected is listed in the VM spec:

   $ oc get vm <vm_name>

Procedure

1. In the Windows Guest Operating System (OS), use the File Explorer to navigate to the guest-agent directory in the virtio-win CD drive.
2. Run the qemu-ga-x86_64.msi installer.

Verification

1. Run the following command to verify that the output contains the QEMU Guest Agent:

   $ net start

Procedure

1. Start the virtual machine and connect to a graphical console.
2. Log in to a Windows user session.

3. Open Device Manager and expand Other devices to list any Unknown device.

   1. Open the Device Properties to identify the unknown device. Right-click the device and select Properties.
   2. Click the Details tab and select Hardware Ids in the Property list.
   3. Compare the Value for the Hardware Ids with the supported VirtIO drivers.
4. Right-click the device and select Update Driver Software.
5. Click Browse my computer for driver software and browse to the attached SATA CD drive, where the VirtIO drivers are located. The drivers are arranged hierarchically according to their driver type, operating system, and CPU architecture.
6. Click Next to install the driver.
7. Repeat this process for all the necessary VirtIO drivers.
8. After the driver installs, click Close to close the window.
9. Reboot the virtual machine to complete the driver installation.

Procedure

1. In the Windows Guest OS, use the File Explorer to navigate to the virtio-win CD drive.

2. Double-click to run the appropriate installer for your VM:

   1. For a 64-bit vCPU, use the virtio-win-gt-x64 installer. 32-bit vCPUs are no longer supported.
3. Optional: During the Custom Setup step of the installer, select the device drivers you want to install. The recommended driver set is selected by default.
4. After the installation is complete, select Finish.
5. Reboot the VM.

Verification

1. Open the system disk on the PC. This is typically C:.
2. Navigate to Program Files Virtio-Win.

Procedure

1. Add the container-native-virtualization/virtio-win container disk as a cdrom disk in the Windows virtual machine configuration file. The container disk will be downloaded from the registry if it is not already present in the cluster.

   spec:
     domain:
       devices:
         disks:
           - name: virtiocontainerdisk
             bootOrder: 2 1
             cdrom:
               bus: sata
   volumes:
     - containerDisk:
         image: container-native-virtualization/virtio-win
       name: virtiocontainerdisk

   1

   OpenShift Virtualization boots virtual machine disks in the order defined in the VirtualMachine configuration file. You can either define other disks for the virtual machine before the container-native-virtualization/virtio-win container disk or use the optional bootOrder parameter to ensure the virtual machine boots from the correct disk. If you specify the bootOrder for a disk, it must be specified for all disks in the configuration.

2. The disk is available once the virtual machine has started:

   • If you add the container disk to a running virtual machine, use oc apply -f <vm.yaml> in the CLI or reboot the virtual machine for the changes to take effect.
   • If the virtual machine is not running, use virtctl start <vm>.

Procedure

1. Click Pipelines Pipelines in the side menu.
2. Select a pipeline to open the Pipeline details page.
3. From the Actions list, select Start. The Start Pipeline dialog is displayed.
4. Keep the default values for the parameters and then click Start to run the pipeline. The Details tab tracks the progress of each task and displays the pipeline status.

Procedure

1. To run the Windows 10 installer pipeline, create the following PipelineRun manifest:

   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     generateName: windows10-installer-run-
     labels:
       pipelinerun: windows10-installer-run
   spec:
     params:
     - name: winImageDownloadURL
       value: <link_to_windows_10_iso> 1
     pipelineRef:
       name: windows10-installer
     taskRunSpecs:
       - pipelineTaskName: copy-template
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template
         taskServiceAccountName: modify-vm-template-task
       - pipelineTaskName: create-vm-from-template
         taskServiceAccountName: create-vm-from-template-task
       - pipelineTaskName: wait-for-vmi-status
         taskServiceAccountName: wait-for-vmi-status-task
       - pipelineTaskName: create-base-dv
         taskServiceAccountName: modify-data-object-task
       - pipelineTaskName: cleanup-vm
         taskServiceAccountName: cleanup-vm-task
     status: {}

   1

   Specify the URL for the Windows 10 64-bit ISO file. The product language must be English (United States).

2. Apply the PipelineRun manifest:

   $ oc apply -f windows10-installer-run.yaml

3. To run the Windows 10 customize pipeline, create the following PipelineRun manifest:

   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     generateName: windows10-customize-run-
     labels:
       pipelinerun: windows10-customize-run
   spec:
     params:
       - name: allowReplaceGoldenTemplate
         value: true
       - name: allowReplaceCustomizationTemplate
         value: true
     pipelineRef:
       name: windows10-customize
     taskRunSpecs:
       - pipelineTaskName: copy-template-customize
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template-customize
         taskServiceAccountName: modify-vm-template-task
       - pipelineTaskName: create-vm-from-template
         taskServiceAccountName: create-vm-from-template-task
       - pipelineTaskName: wait-for-vmi-status
         taskServiceAccountName: wait-for-vmi-status-task
       - pipelineTaskName: create-base-dv
         taskServiceAccountName: modify-data-object-task
       - pipelineTaskName: cleanup-vm
         taskServiceAccountName: cleanup-vm-task
       - pipelineTaskName: copy-template-golden
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template-golden
         taskServiceAccountName: modify-vm-template-task
   status: {}

4. Apply the PipelineRun manifest:

   $ oc apply -f windows10-customize-run.yaml

Procedure

1. Set limits for a VM by editing the VirtualMachine manifest. For example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: with-limits
   spec:
     running: false
     template:
       spec:
         domain:
   # ...
           resources:
             requests:
               memory: 128Mi
             limits:
               memory: 256Mi  1

   1

   This configuration is supported because the limits.memory value is at least 100Mi larger than the requests.memory value.

2. Save the VirtualMachine manifest.

Procedure

1. Open the HyperConverged CR by running the following command:

   $ oc edit hco -n openshift-cnv kubevirt-hyperconverged

2. Edit the spec.certConfig fields as shown in the following example. To avoid overloading the system, ensure that all values are greater than or equal to 10 minutes. Express all values as strings that comply with the golang ParseDuration format.

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
    name: kubevirt-hyperconverged
    namespace: openshift-cnv
   spec:
     certConfig:
       ca:
         duration: 48h0m0s
         renewBefore: 24h0m0s 1
       server:
         duration: 24h0m0s  2
         renewBefore: 12h0m0s  3

   1

   The value of ca.renewBefore must be less than or equal to the value of ca.duration.

   2

   The value of server.duration must be less than or equal to the value of ca.duration.

   3

   The value of server.renewBefore must be less than or equal to the value of server.duration.

3. Apply the YAML file to your cluster.

Procedure

1. Open the HyperConverged CR by running the following command:

   $ oc edit hco -n openshift-cnv kubevirt-hyperconverged

2. Add the defaultCPUModel field to the CR and set the value to the name of a CPU model that exists in the cluster:

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
    name: kubevirt-hyperconverged
    namespace: openshift-cnv
   spec:
     defaultCPUModel: "EPYC"

3. Apply the YAML file to your cluster.

Procedure

1. Edit or create a VirtualMachine manifest file. Use the spec.firmware.bootloader stanza to configure UEFI mode:

   Booting in UEFI mode with secure boot active

   apiversion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     labels:
       special: vm-secureboot
     name: vm-secureboot
   spec:
     template:
       metadata:
         labels:
           special: vm-secureboot
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: containerdisk
           features:
             acpi: {}
             smm:
               enabled: true 1
           firmware:
             bootloader:
               efi:
                 secureBoot: true 2
   # ...

   1

   OpenShift Virtualization requires System Management Mode (SMM) to be enabled for Secure Boot in UEFI mode to occur.

   2

   OpenShift Virtualization supports a VM with or without Secure Boot when using UEFI mode. If Secure Boot is enabled, then UEFI mode is required. However, UEFI mode can be enabled without using Secure Boot.

2. Apply the manifest to your cluster by running the following command:

   $ oc create -f <file_name>.yaml

Procedure

1. Configure a PXE network on the cluster:

   1. Create the network attachment definition file for PXE network pxe-net-conf:

      apiVersion: "k8s.cni.cncf.io/v1"
      kind: NetworkAttachmentDefinition
      metadata:
        name: pxe-net-conf
      spec:
        config: '{
          "cniVersion": "0.3.1",
          "name": "pxe-net-conf",
          "plugins": [
            {
              "type": "cnv-bridge",
              "bridge": "br1",
              "vlan": 1 1
            },
            {
              "type": "cnv-tuning" 2
            }
          ]
        }'

      1

      Optional: The VLAN tag.

      2

      The cnv-tuning plugin provides support for custom MAC addresses.

      Note

      The virtual machine instance will be attached to the bridge br1 through an access port with the requested VLAN.

2. Create the network attachment definition by using the file you created in the previous step:

   $ oc create -f pxe-net-conf.yaml

3. Edit the virtual machine instance configuration file to include the details of the interface and network.

   1. Specify the network and MAC address, if required by the PXE server. If the MAC address is not specified, a value is assigned automatically.

      Ensure that bootOrder is set to 1 so that the interface boots first. In this example, the interface is connected to a network called <pxe-net>:

      interfaces:
      - masquerade: {}
        name: default
      - bridge: {}
        name: pxe-net
        macAddress: de:00:00:00:00:de
        bootOrder: 1

      Note

      Boot order is global for interfaces and disks.

   2. Assign a boot device number to the disk to ensure proper booting after operating system provisioning.

      Set the disk bootOrder value to 2:

      devices:
        disks:
        - disk:
            bus: virtio
          name: containerdisk
          bootOrder: 2

   3. Specify that the network is connected to the previously created network attachment definition. In this scenario, <pxe-net> is connected to the network attachment definition called <pxe-net-conf>:

      networks:
      - name: default
        pod: {}
      - name: pxe-net
        multus:
          networkName: pxe-net-conf

4. Create the virtual machine instance:

   $ oc create -f vmi-pxe-boot.yaml

1. Wait for the virtual machine instance to run:

   $ oc get vmi vmi-pxe-boot -o yaml | grep -i phase
     phase: Running

2. View the virtual machine instance using VNC:

   $ virtctl vnc vmi-pxe-boot

3. Watch the boot screen to verify that the PXE boot is successful.

4. Log in to the virtual machine instance:

   $ virtctl console vmi-pxe-boot

5. Verify the interfaces and MAC address on the virtual machine and that the interface connected to the bridge has the specified MAC address. In this case, we used eth1 for the PXE boot, without an IP address. The other interface, eth0, got an IP address from OpenShift Container Platform.

   $ ip addr

Procedure

1. In your virtual machine configuration, add the resources.requests.memory and memory.hugepages.pageSize parameters to the spec.domain. The following configuration snippet is for a virtual machine that requests a total of 4Gi memory with a page size of 1Gi:

   kind: VirtualMachine
   # ...
   spec:
     domain:
       resources:
         requests:
           memory: "4Gi" 1
       memory:
         hugepages:
           pageSize: "1Gi" 2
   # ...

   1

   The total amount of memory requested for the virtual machine. This value must be divisible by the page size.

   2

   The size of each huge page. Valid values for x86_64 architecture are 1Gi and 2Mi. The page size must be smaller than the requested memory.

2. Apply the virtual machine configuration:

   $ oc apply -f <virtual_machine>.yaml

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Select a virtual machine to open the VirtualMachine details page.
3. On the Configuration Scheduling tab, click the edit icon beside Dedicated Resources.
4. Select Schedule this workload with dedicated resources (guaranteed policy).
5. Click Save.

Procedure

1. In the OpenShift Container Platform web console, click Virtualization VirtualMachines from the side menu.
2. Select the virtual machine to which you want to assign the device.

3. On the Details tab, click GPU devices.

   If you add a vGPU device as a host device, you cannot access the device with the VNC console.

4. Click Add GPU device, enter the Name and select the device from the Device name list.
5. Click Save.
6. Click the YAML tab to verify that the new devices have been added to your cluster configuration in the hostDevices section.

Procedure

1. Log in to the OpenShift Container Platform web console.

2. Create the required namespace for the Kube Descheduler Operator.

   1. Navigate to Administration Namespaces and click Create Namespace.
   2. Enter openshift-kube-descheduler-operator in the Name field, enter openshift.io/cluster-monitoring=true in the Labels field to enable descheduler metrics, and click Create.

3. Install the Kube Descheduler Operator.

   1. Navigate to Operators OperatorHub.
   2. Type Kube Descheduler Operator into the filter box.
   3. Select the Kube Descheduler Operator and click Install.
   4. On the Install Operator page, select A specific namespace on the cluster. Select openshift-kube-descheduler-operator from the drop-down menu.
   5. Adjust the values for the Update Channel and Approval Strategy to the desired values.
   6. Click Install.

4. Create a descheduler instance.

   1. From the Operators Installed Operators page, click the Kube Descheduler Operator.
   2. Select the Kube Descheduler tab and click Create KubeDescheduler.

   3. Edit the settings as necessary.

      1. To evict pods instead of simulating the evictions, change the Mode field to Automatic.

      2. Expand the Profiles section and select DevPreviewLongLifecycle. The AffinityAndTaints profile is enabled by default.

         Important

         The only profile currently available for OpenShift Virtualization is DevPreviewLongLifecycle.

Procedure

1. Before starting the VM, add the descheduler.alpha.kubernetes.io/evict annotation to the VirtualMachine CR:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   spec:
     template:
       metadata:
         annotations:
           descheduler.alpha.kubernetes.io/evict: "true"

2. If you did not already set the DevPreviewLongLifecycle profile in the web console during installation, specify the DevPreviewLongLifecycle in the spec.profile section of the KubeDescheduler object:

   apiVersion: operator.openshift.io/v1
   kind: KubeDescheduler
   metadata:
     name: cluster
     namespace: openshift-kube-descheduler-operator
   spec:
     deschedulingIntervalSeconds: 3600
     profiles:
     - DevPreviewLongLifecycle
     mode: Predictive 1

   1

   By default, the descheduler does not evict pods. To evict pods, set mode to Automatic.

Procedure

1. Ensure you are in the correct namespace. The config map can only be referenced by data volumes if it is in the same namespace.

   $ oc get ns

2. Create the config map:

   $ oc create configmap <configmap-name> --from-file=</path/to/file/ca.pem>

Procedure

1. If your data source requires authentication, create a Secret manifest, specifying the data source credentials, and save it as endpoint-secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
     name: endpoint-secret 1
     labels:
       app: containerized-data-importer
   type: Opaque
   data:
     accessKeyId: "" 2
     secretKey:   "" 3

   1

   Specify the name of the Secret.

   2

   Specify the Base64-encoded key ID or user name.

   3

   Specify the Base64-encoded secret key or password.

2. Apply the Secret manifest:

   $ oc apply -f endpoint-secret.yaml

3. Edit the VirtualMachine manifest, specifying the data source for the virtual machine image you want to import, and save it as vm-fedora-datavolume.yaml:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     creationTimestamp: null
     labels:
       kubevirt.io/vm: vm-fedora-datavolume
     name: vm-fedora-datavolume 1
   spec:
     dataVolumeTemplates:
     - metadata:
         creationTimestamp: null
         name: fedora-dv 2
       spec:
         storage:
           volumeMode: Block 3
           resources:
             requests:
               storage: 10Gi
           storageClassName: local
         source: 4
           http:
             url: "https://mirror.arizona.edu/fedora/linux/releases/35/Cloud/x86_64/images/Fedora-Cloud-Base-35-1.2.x86_64.qcow2" 5
             secretRef: endpoint-secret 6
             certConfigMap: "" 7
           # To use a registry source, uncomment the following lines and delete the preceding HTTP source block
           # registry:
             # url: "docker://kubevirt/fedora-cloud-container-disk-demo:latest"
             # secretRef: registry-secret 8
             # certConfigMap: "" 9
       status: {}
     running: true
     template:
       metadata:
         creationTimestamp: null
         labels:
           kubevirt.io/vm: vm-fedora-datavolume
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: datavolumedisk1
           machine:
             type: ""
           resources:
             requests:
               memory: 1.5Gi
         terminationGracePeriodSeconds: 180
         volumes:
         - dataVolume:
             name: fedora-dv
           name: datavolumedisk1
   status: {}

   1

   Specify the name of the virtual machine.

   2

   Specify the name of the data volume.

   3

   The volume and access mode are detected automatically for known storage provisioners. Alternatively, you can specify Block.

   4 5

   Specify either the URL or the registry endpoint of the virtual machine image you want to import using the comment block. For example, if you want to use a registry source, you can comment out or delete the HTTP or HTTPS source block. Ensure that you replace the example values shown here with your own values.

   6 8

   Specify the Secret name if you created a Secret for the data source.

   7 9

   Optional: Specify a CA certificate config map.

4. Create the virtual machine:

   $ oc create -f vm-fedora-datavolume.yaml

   Note

   The oc create command creates the data volume and the virtual machine. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes to Succeeded. You can start the virtual machine.

   Data volume provisioning happens in the background, so there is no need to monitor the process.

Verification

1. The importer pod downloads the virtual machine image or container disk from the specified URL and stores it on the provisioned PV. View the status of the importer pod by running the following command:

   $ oc get pods

2. Monitor the data volume until its status is Succeeded by running the following command:

   $ oc describe dv fedora-dv 1

   1

   Specify the data volume name that you defined in the VirtualMachine manifest.

3. Verify that provisioning is complete and that the virtual machine has started by accessing its serial console:

   $ virtctl console vm-fedora-datavolume

Procedure

1. Create a ClusterRole manifest:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: <datavolume-cloner> 1
   rules:
   - apiGroups: ["cdi.kubevirt.io"]
     resources: ["datavolumes/source"]
     verbs: ["*"]

   1

   Unique name for the cluster role.

2. Create the cluster role in the cluster:

   $ oc create -f <datavolume-cloner.yaml> 1

   1

   The file name of the ClusterRole manifest created in the previous step.

3. Create a RoleBinding manifest that applies to both the source and destination namespaces and references the cluster role created in the previous step.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: <allow-clone-to-user> 1
     namespace: <Source namespace> 2
   subjects:
   - kind: ServiceAccount
     name: default
     namespace: <Destination namespace> 3
   roleRef:
     kind: ClusterRole
     name: datavolume-cloner 4
     apiGroup: rbac.authorization.k8s.io

   1

   Unique name for the role binding.

   2

   The namespace for the source data volume.

   3

   The namespace to which the data volume is cloned.

   4

   The name of the cluster role created in the previous step.

4. Create the role binding in the cluster:

   $ oc create -f <datavolume-cloner.yaml> 1

   1

   The file name of the RoleBinding manifest created in the previous step.

Procedure

1. Examine the virtual machine disk you want to clone to identify the name and namespace of the associated PVC.

2. Create a YAML file for a data volume that specifies the name of the new data volume, the name and namespace of the source PVC, and the size of the new data volume.

   For example:

   apiVersion: cdi.kubevirt.io/v1beta1
   kind: DataVolume
   metadata:
     name: <cloner-datavolume> 1
   spec:
     source:
       pvc:
         namespace: "<source-namespace>" 2
         name: "<my-favorite-vm-disk>" 3
     pvc:
       accessModes:
         - ReadWriteOnce
       resources:
         requests:
           storage: <2Gi> 4

   1

   The name of the new data volume.

   2

   The namespace where the source PVC exists.

   3

   The name of the source PVC.

   4

   The size of the new data volume. You must allocate enough space, or the cloning operation fails. The size must be the same as or larger than the source PVC.

3. Start cloning the PVC by creating the data volume:

   $ oc create -f <cloner-datavolume>.yaml

   Note

   Data volumes prevent a virtual machine from starting before the PVC is prepared, so you can create a virtual machine that references the new data volume while the PVC clones.

Procedure

1. Examine the virtual machine you want to clone to identify the name and namespace of the associated PVC.

2. Create a YAML file for a VirtualMachine object. The following virtual machine example clones my-favorite-vm-disk, which is located in the source-namespace namespace. The 2Gi data volume called favorite-clone is created from my-favorite-vm-disk.

   For example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     labels:
       kubevirt.io/vm: vm-dv-clone
     name: vm-dv-clone 1
   spec:
     running: false
     template:
       metadata:
         labels:
           kubevirt.io/vm: vm-dv-clone
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: root-disk
           resources:
             requests:
               memory: 64M
         volumes:
         - dataVolume:
             name: favorite-clone
           name: root-disk
     dataVolumeTemplates:
     - metadata:
         name: favorite-clone
       spec:
         storage:
           accessModes:
           - ReadWriteOnce
           resources:
             requests:
               storage: 2Gi
         source:
           pvc:
             namespace: "source-namespace"
             name: "my-favorite-vm-disk"

   1

   The virtual machine to create.

3. Create the virtual machine with the PVC-cloned data volume:

   $ oc create -f <vm-clone-datavolumetemplate>.yaml

Procedure

1. Log in as root to the node on which to create the local PV. This procedure uses node01 for its examples.

2. Create a file and populate it with null characters so that it can be used as a block device. The following example creates a file loop10 with a size of 2Gb (20 100Mb blocks):

   $ dd if=/dev/zero of=<loop10> bs=100M count=20

3. Mount the loop10 file as a loop device.

   $ losetup </dev/loop10>d3 <loop10> 1 2

   1

   File path where the loop device is mounted.

   2

   The file created in the previous step to be mounted as the loop device.

4. Create a PersistentVolume manifest that references the mounted loop device.

   kind: PersistentVolume
   apiVersion: v1
   metadata:
     name: <local-block-pv10>
     annotations:
   spec:
     local:
       path: </dev/loop10> 1
     capacity:
       storage: <2Gi>
     volumeMode: Block 2
     storageClassName: local 3
     accessModes:
       - ReadWriteOnce
     persistentVolumeReclaimPolicy: Delete
     nodeAffinity:
       required:
         nodeSelectorTerms:
         - matchExpressions:
           - key: kubernetes.io/hostname
             operator: In
             values:
             - <node01> 4

   1

   The path of the loop device on the node.

   2

   Specifies it is a block PV.

   3

   Optional: Set a storage class for the PV. If you omit it, the cluster default is used.

   4

   The node on which the block device was mounted.

5. Create the block PV.

   # oc create -f <local-block-pv10.yaml>1

   1

   The file name of the persistent volume created in the previous step.

Procedure

1. Examine the virtual machine disk you want to clone to identify the name and namespace of the associated PVC.

2. Create a YAML file for a data volume that specifies the name of the new data volume, the name and namespace of the source PVC, volumeMode: Block so that an available block PV is used, and the size of the new data volume.

   For example:

   apiVersion: cdi.kubevirt.io/v1beta1
   kind: DataVolume
   metadata:
     name: <cloner-datavolume> 1
   spec:
     source:
       pvc:
         namespace: "<source-namespace>" 2
         name: "<my-favorite-vm-disk>" 3
     pvc:
       accessModes:
         - ReadWriteOnce
       resources:
         requests:
           storage: <2Gi> 4
       volumeMode: Block 5

   1

   The name of the new data volume.

   2

   The namespace where the source PVC exists.

   3

   The name of the source PVC.

   4

   The size of the new data volume. You must allocate enough space, or the cloning operation fails. The size must be the same as or larger than the source PVC.

   5

   Specifies that the destination is a block PV

3. Start cloning the PVC by creating the data volume:

   $ oc create -f <cloner-datavolume>.yaml

   Note

   Data volumes prevent a virtual machine from starting before the PVC is prepared, so you can create a virtual machine that references the new data volume while the PVC clones.

Procedure

1. Edit the interfaces spec of your virtual machine configuration file:

   kind: VirtualMachine
   spec:
     domain:
       devices:
         interfaces:
           - name: default
             masquerade: {} 1
             ports: 2
               - port: 80
     networks:
     - name: default
       pod: {}

   1

   Connect using masquerade mode.

   2

   Optional: List the ports that you want to expose from the virtual machine, each specified by the port field. The port value must be a number between 0 and 65536. When the ports array is not used, all ports in the valid range are open to incoming traffic. In this example, incoming traffic is allowed on port 80.

   Note

   Ports 49152 and 49153 are reserved for use by the libvirt platform and all other incoming traffic to these ports is dropped.

2. Create the virtual machine:

   $ oc create -f <vm-name>.yaml

Procedure

1. In a new virtual machine configuration, include an interface with masquerade and configure the IPv6 address and default gateway by using cloud-init.

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: example-vm-ipv6
   # ...
             interfaces:
               - name: default
                 masquerade: {} 1
                 ports:
                   - port: 80 2
         networks:
         - name: default
           pod: {}
         volumes:
         - cloudInitNoCloud:
             networkData: |
               version: 2
               ethernets:
                 eth0:
                   dhcp4: true
                   addresses: [ fd10:0:2::2/120 ] 3
                   gateway6: fd10:0:2::1 4

   1

   Connect using masquerade mode.

   2

   Allows incoming traffic on port 80 to the virtual machine.

   3

   The static IPv6 address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::2/120.

   4

   The gateway IP address as determined by the Network.pod.vmIPv6NetworkCIDR field in the virtual machine instance configuration. The default value is fd10:0:2::1.

2. Create the virtual machine in the namespace:

   $ oc create -f example-vm-ipv6.yaml

Procedure

1. Edit the VirtualMachine manifest to add the label for service creation:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: vm-ephemeral
     namespace: example-namespace
   spec:
     running: false
     template:
       metadata:
         labels:
           special: key 1
   # ...

   1

   Add the label special: key in the spec.template.metadata.labels section.

   Note

   Labels on a virtual machine are passed through to the pod. The special: key label must match the label in the spec.selector attribute of the Service manifest.

2. Save the VirtualMachine manifest file to apply your changes.

3. Create a Service manifest to expose the VM:

   apiVersion: v1
   kind: Service
   metadata:
     name: vmservice 1
     namespace: example-namespace 2
   spec:
     externalTrafficPolicy: Cluster 3
     ports:
     - nodePort: 30000 4
       port: 27017
       protocol: TCP
       targetPort: 22 5
     selector:
       special: key 6
     type: NodePort 7

   1

   The name of the Service object.

   2

   The namespace where the Service object resides. This must match the metadata.namespace field of the VirtualMachine manifest.

   3

   Optional: Specifies how the nodes distribute service traffic that is received on external IP addresses. This only applies to NodePort and LoadBalancer service types. The default value is Cluster which routes traffic evenly to all cluster endpoints.

   4

   Optional: When set, the nodePort value must be unique across all services. If not specified, a value in the range above 30000 is dynamically allocated.

   5

   Optional: The VM port to be exposed by the service. It must reference an open port if a port list is defined in the VM manifest. If targetPort is not specified, it takes the same value as port.

   6

   The reference to the label that you added in the spec.template.metadata.labels stanza of the VirtualMachine manifest.

   7

   The type of service. Possible values are ClusterIP, NodePort and LoadBalancer.

4. Save the Service manifest file.

5. Create the service by running the following command:

   $ oc create -f <service_name>.yaml

6. Start the VM. If the VM is already running, restart it.

Verification

1. Query the Service object to verify that it is available:

   $ oc get service -n example-namespace

   Example output for ClusterIP service

   NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
   vmservice   ClusterIP   172.30.3.149   <none>        27017/TCP   2m

   Example output for NodePort service

   NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)            AGE
   vmservice   NodePort    172.30.232.73   <none>       27017:30000/TCP    5m

   Example output for LoadBalancer service

   NAME        TYPE            CLUSTER-IP     EXTERNAL-IP                    PORT(S)           AGE
   vmservice   LoadBalancer    172.30.27.5   172.29.10.235,172.29.10.235     27017:31829/TCP   5s

2. Choose the appropriate method to connect to the virtual machine:

   • For a ClusterIP service, connect to the VM from within the cluster by using the service IP address and the service port. For example:

     $ ssh fedora@172.30.3.149 -p 27017

   • For a NodePort service, connect to the VM by specifying the node IP address and the node port outside the cluster network. For example:

     $ ssh fedora@$NODE_IP -p 30000

   • For a LoadBalancer service, use the vinagre client to connect to your virtual machine by using the public IP address and port. External ports are dynamically allocated.

Procedure

1. Create an SriovNetworkNodePolicy object, and then save the YAML in the <name>-sriov-node-network.yaml file. Replace <name> with the name for this configuration.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: <name> 1
     namespace: openshift-sriov-network-operator 2
   spec:
     resourceName: <sriov_resource_name> 3
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true" 4
     priority: <priority> 5
     mtu: <mtu> 6
     numVfs: <num> 7
     nicSelector: 8
       vendor: "<vendor_code>" 9
       deviceID: "<device_id>" 10
       pfNames: ["<pf_name>", ...] 11
       rootDevices: ["<pci_bus_id>", "..."] 12
     deviceType: vfio-pci 13
     isRdma: false 14

   1

   Specify a name for the CR object.

   2

   Specify the namespace where the SR-IOV Operator is installed.

   3

   Specify the resource name of the SR-IOV device plugin. You can create multiple SriovNetworkNodePolicy objects for a resource name.

   4

   Specify the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.

   5

   Optional: Specify an integer value between 0 and 99. A smaller number gets higher priority, so a priority of 10 is higher than a priority of 99. The default value is 99.

   6

   Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.

   7

   Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than 127.

   8

   The nicSelector mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters. It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally. If you specify rootDevices, you must also specify a value for vendor, deviceID, or pfNames. If you specify both pfNames and rootDevices at the same time, ensure that they point to an identical device.

   9

   Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either 8086 or 15b3.

   10

   Optional: Specify the device hex code of SR-IOV network device. The only allowed values are 158b, 1015, 1017.

   11

   Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.

   12

   The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: 0000:02:00.1.

   13

   The vfio-pci driver type is required for virtual functions in OpenShift Virtualization.

   14

   Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set isRdma to false. The default value is false.

   Note

   If isRDMA flag is set to true, you can continue to use the RDMA enabled VF as a normal network device. A device can be used in either mode.

2. Optional: Label the SR-IOV capable cluster nodes with SriovNetworkNodePolicy.Spec.NodeSelector if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".

3. Create the SriovNetworkNodePolicy object:

   $ oc create -f <name>-sriov-node-network.yaml

   where <name> specifies the name for this configuration.

   After applying the configuration update, all the pods in sriov-network-operator namespace transition to the Running status.

4. To verify that the SR-IOV network device is configured, enter the following command. Replace <node_name> with the name of a node with the SR-IOV network device that you just configured.

   $ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'

Procedure

1. Create the following SriovNetwork object, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

2. To create the object, enter the following command. Replace <name> with a name for this additional network.

   $ oc create -f <name>-sriov-network.yaml

3. Optional: To confirm that the NetworkAttachmentDefinition object associated with the SriovNetwork object that you created in the previous step exists, enter the following command. Replace <namespace> with the namespace you specified in the SriovNetwork object.

   $ oc get net-attach-def -n <namespace>

Procedure

1. Include the SR-IOV network details in the spec.domain.devices.interfaces and spec.networks of the VM configuration:

   kind: VirtualMachine
   # ...
   spec:
     domain:
       devices:
         interfaces:
         - name: <default> 1
           masquerade: {} 2
         - name: <nic1> 3
           sriov: {}
     networks:
     - name: <default> 4
       pod: {}
     - name: <nic1> 5
       multus:
           networkName: <sriov-network> 6
   # ...

   1

   A unique name for the interface that is connected to the pod network.

   2

   The masquerade binding to the default pod network.

   3

   A unique name for the SR-IOV interface.

   4

   The name of the pod network interface. This must be the same as the interfaces.name that you defined earlier.

   5

   The name of the SR-IOV interface. This must be the same as the interfaces.name that you defined earlier.

   6

   The name of the SR-IOV network attachment definition.

2. Apply the virtual machine configuration:

   $ oc apply -f <vm-sriov.yaml> 1

   1

   The name of the virtual machine YAML file.