Chapter 7. Virtual machines

Procedure

1. In the web console, navigate to Virtualization Catalog and click the InstanceTypes tab.

2. Select either of the following options:

   • Select a bootable volume.

     Note

     The bootable volume table lists only those volumes in the openshift-virtualization-os-images namespace that have the instancetype.kubevirt.io/default-preference label.

     • Optional: Click the star icon to designate a bootable volume as a favorite. Starred bootable volumes appear first in the volume list.
   • Click Add volume to upload a new volume or use an existing persistent volume claim (PVC), volume snapshot, or data source. Then click Save.
3. Click an instance type tile and select the resource size appropriate for your workload.
4. If you have not already added a public SSH key to your project, click the edit icon beside Authorized SSH key in the VirtualMachine details section.

5. Select one of the following options:

   • Use existing: Select a secret from the secrets list.

   • Add new:

     1. Browse to the public SSH key file or paste the file in the key field.
     2. Enter the secret name.
     3. Optional: Select Automatically apply this key to any new VirtualMachine you create in this project.
     4. Click Save.
6. Optional: Click View YAML & CLI to view the YAML file. Click CLI to view the CLI commands. You can also download or copy either the YAML file contents or the CLI commands.
7. Click Create VirtualMachine.

Procedure

1. Navigate to Virtualization Catalog in the web console.

2. Click Boot source available to filter templates with boot sources.

   The catalog displays the default templates. Click All Items to view all available templates for your filters.

3. Click a template tile to view its details.

4. Click Quick create VirtualMachine to create a VM from the template.

   Optional: Customize the template or VM parameters:

   1. Click Customize VirtualMachine.
   2. Expand Storage or Optional parameters to edit data source settings.

   3. Click Customize VirtualMachine parameters.

      The Customize and create VirtualMachine pane displays the Overview, YAML, Scheduling, Environment, Network interfaces, Disks, Scripts, and Metadata tabs.

   4. Edit the parameters that must be set before the VM boots, such as cloud-init or a static SSH key.

   5. Click Create VirtualMachine.

      The VirtualMachine details page displays the provisioning status.

Procedure

1. Edit the VirtualMachine manifest for your VM. The following example configures a Red Hat Enterprise Linux (RHEL) VM:

   Note

   This example manifest does not configure VM authentication.

   Example manifest for a RHEL VM

    apiVersion: kubevirt.io/v1
    kind: VirtualMachine
    metadata:
     name: rhel-9-minimal
    spec:
     dataVolumeTemplates:
       - metadata:
           name: rhel-9-minimal-volume
         spec:
           sourceRef:
             kind: DataSource
             name: rhel9 1
             namespace: openshift-virtualization-os-images 2
           storage: {}
     instancetype:
       name: u1.medium 3
     preference:
       name: rhel.9 4
     running: true
     template:
       spec:
         domain:
           devices: {}
         volumes:
           - dataVolume:
               name: rhel-9-minimal-volume
             name: rootdisk

   1

   The rhel9 golden image is used to install RHEL 9 as the guest operating system.

   2

   Golden images are stored in the openshift-virtualization-os-images namespace.

   3

   The u1.medium instance type requests 1 vCPU and 4Gi memory for the VM. These resource values cannot be overridden within the VM.

   4

   The rhel.9 preference specifies additional attributes that support the RHEL 9 guest operating system.

2. Create a virtual machine by using the manifest file:

   $ oc create -f <vm_manifest_file>.yaml

3. Optional: Start the virtual machine:

   $ virtctl start <vm_name> -n <namespace>

Procedure

1. Create a Dockerfile to build the VM image into a container image. The VM image must be owned by QEMU, which has a UID of 107, and placed in the /disk/ directory inside the container. Permissions for the /disk/ directory must then be set to 0440.

   The following example uses the Red Hat Universal Base Image (UBI) to handle these configuration changes in the first stage, and uses the minimal scratch image in the second stage to store the result:

   $ cat > Dockerfile << EOF
   FROM registry.access.redhat.com/ubi8/ubi:latest AS builder
   ADD --chown=107:107 <vm_image>.qcow2 /disk/ 1
   RUN chmod 0440 /disk/*
   
   FROM scratch
   COPY --from=builder /disk/* /disk/
   EOF

   1

   Where <vm_image> is the image in either QCOW2 or RAW format. If you use a remote image, replace <vm_image>.qcow2 with the complete URL.

2. Build and tag the container:

   $ podman build -t <registry>/<container_disk_name>:latest .

3. Push the container image to the registry:

   $ podman push <registry>/<container_disk_name>:latest

Prerequisites

1. Open the HyperConverged CR in your default editor by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

2. Add a list of insecure registries to the spec.storageImport.insecureRegistries field.

   Example HyperConverged custom resource

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     storageImport:
       insecureRegistries: 1
         - "private-registry-example-1:5000"
         - "private-registry-example-2:5000"

   1

   Replace the examples in this list with valid registry hostnames.

Procedure

1. Navigate to Virtualization Catalog in the web console.
2. Click a template tile without an available boot source.
3. Click Customize VirtualMachine.
4. On the Customize template parameters page, expand Storage and select Registry (creates PVC) from the Disk source list.
5. Enter the container image URL. Example: https://mirror.arizona.edu/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2
6. Set the disk size.
7. Click Next.
8. Click Create VirtualMachine.

Procedure

1. If the container registry requires authentication, create a Secret manifest, specifying the credentials, and save it as a data-source-secret.yaml file:

   apiVersion: v1
   kind: Secret
   metadata:
     name: data-source-secret
     labels:
       app: containerized-data-importer
   type: Opaque
   data:
     accessKeyId: "" 1
     secretKey:   "" 2

   1

   Specify the Base64-encoded key ID or user name.

   2

   Specify the Base64-encoded secret key or password.

2. Apply the Secret manifest by running the following command:

   $ oc apply -f data-source-secret.yaml

3. If the VM must communicate with servers that use self-signed certificates or certificates that are not signed by the system CA bundle, create a config map in the same namespace as the VM:

   $ oc create configmap tls-certs 1
     --from-file=</path/to/file/ca.pem> 2

   1

   Specify the config map name.

   2

   Specify the path to the CA certificate.

4. Edit the VirtualMachine manifest and save it as a vm-fedora-datavolume.yaml file:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     creationTimestamp: null
     labels:
       kubevirt.io/vm: vm-fedora-datavolume
     name: vm-fedora-datavolume 1
   spec:
     dataVolumeTemplates:
     - metadata:
         creationTimestamp: null
         name: fedora-dv 2
       spec:
         storage:
           resources:
             requests:
               storage: 10Gi 3
           storageClassName: <storage_class> 4
         source:
           registry:
             url: "docker://kubevirt/fedora-cloud-container-disk-demo:latest" 5
             secretRef: data-source-secret 6
             certConfigMap: tls-certs 7
       status: {}
     running: true
     template:
       metadata:
         creationTimestamp: null
         labels:
           kubevirt.io/vm: vm-fedora-datavolume
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: datavolumedisk1
           machine:
             type: ""
           resources:
             requests:
               memory: 1.5Gi
         terminationGracePeriodSeconds: 180
         volumes:
         - dataVolume:
             name: fedora-dv
           name: datavolumedisk1
   status: {}

   1

   Specify the name of the VM.

   2

   Specify the name of the data volume.

   3

   Specify the size of the storage requested for the data volume.

   4

   Optional: If you do not specify a storage class, the default storage class is used.

   5

   Specify the URL of the container registry.

   6

   Optional: Specify the secret name if you created a secret for the container registry access credentials.

   7

   Optional: Specify a CA certificate config map.

5. Create the VM by running the following command:

   $ oc create -f vm-fedora-datavolume.yaml

   The oc create command creates the data volume and the VM. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes to Succeeded. You can start the VM.

   Data volume provisioning happens in the background, so there is no need to monitor the process.

Verification

1. The importer pod downloads the container disk from the specified URL and stores it on the provisioned persistent volume. View the status of the importer pod by running the following command:

   $ oc get pods

2. Monitor the data volume until its status is Succeeded by running the following command:

   $ oc describe dv fedora-dv 1

   1

   Specify the data volume name that you defined in the VirtualMachine manifest.

3. Verify that provisioning is complete and that the VM has started by accessing its serial console:

   $ virtctl console vm-fedora-datavolume

Procedure

1. Navigate to Virtualization Catalog in the web console.
2. Click a template tile without an available boot source.
3. Click Customize VirtualMachine.
4. On the Customize template parameters page, expand Storage and select URL (creates PVC) from the Disk source list.
5. Enter the image URL. Example: https://access.redhat.com/downloads/content/69/ver=/rhel---7/7.9/x86_64/product-software
6. Enter the container image URL. Example: https://mirror.arizona.edu/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2
7. Set the disk size.
8. Click Next.
9. Click Create VirtualMachine.

Procedure

1. If the web page requires authentication, create a Secret manifest, specifying the credentials, and save it as a data-source-secret.yaml file:

   apiVersion: v1
   kind: Secret
   metadata:
     name: data-source-secret
     labels:
       app: containerized-data-importer
   type: Opaque
   data:
     accessKeyId: "" 1
     secretKey:   "" 2

   1

   Specify the Base64-encoded key ID or user name.

   2

   Specify the Base64-encoded secret key or password.

2. Apply the Secret manifest by running the following command:

   $ oc apply -f data-source-secret.yaml

3. If the VM must communicate with servers that use self-signed certificates or certificates that are not signed by the system CA bundle, create a config map in the same namespace as the VM:

   $ oc create configmap tls-certs 1
     --from-file=</path/to/file/ca.pem> 2

   1

   Specify the config map name.

   2

   Specify the path to the CA certificate.

4. Edit the VirtualMachine manifest and save it as a vm-fedora-datavolume.yaml file:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     creationTimestamp: null
     labels:
       kubevirt.io/vm: vm-fedora-datavolume
     name: vm-fedora-datavolume 1
   spec:
     dataVolumeTemplates:
     - metadata:
         creationTimestamp: null
         name: fedora-dv 2
       spec:
         storage:
           resources:
             requests:
               storage: 10Gi 3
           storageClassName: <storage_class> 4
         source:
           http:
             url: "https://mirror.arizona.edu/fedora/linux/releases/35/Cloud/x86_64/images/Fedora-Cloud-Base-35-1.2.x86_64.qcow2" 5
           registry:
             url: "docker://kubevirt/fedora-cloud-container-disk-demo:latest" 6
             secretRef: data-source-secret 7
             certConfigMap: tls-certs 8
       status: {}
     running: true
     template:
       metadata:
         creationTimestamp: null
         labels:
           kubevirt.io/vm: vm-fedora-datavolume
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: datavolumedisk1
           machine:
             type: ""
           resources:
             requests:
               memory: 1.5Gi
         terminationGracePeriodSeconds: 180
         volumes:
         - dataVolume:
             name: fedora-dv
           name: datavolumedisk1
   status: {}

   1

   Specify the name of the VM.

   2

   Specify the name of the data volume.

   3

   Specify the size of the storage requested for the data volume.

   4

   Optional: If you do not specify a storage class, the default storage class is used.

   5 6

   Specify the URL of the web page.

   7

   Optional: Specify the secret name if you created a secret for the web page access credentials.

   8

   Optional: Specify a CA certificate config map.

5. Create the VM by running the following command:

   $ oc create -f vm-fedora-datavolume.yaml

   The oc create command creates the data volume and the VM. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes to Succeeded. You can start the VM.

   Data volume provisioning happens in the background, so there is no need to monitor the process.

Verification

1. The importer pod downloads the image from the specified URL and stores it on the provisioned persistent volume. View the status of the importer pod by running the following command:

   $ oc get pods

2. Monitor the data volume until its status is Succeeded by running the following command:

   $ oc describe dv fedora-dv 1

   1

   Specify the data volume name that you defined in the VirtualMachine manifest.

3. Verify that provisioning is complete and that the VM has started by accessing its serial console:

   $ virtctl console vm-fedora-datavolume

Procedure

1. Navigate to Virtualization Catalog in the web console.
2. Click a template tile without an available boot source.
3. Click Customize VirtualMachine.
4. On the Customize template parameters page, expand Storage and select Upload (Upload a new file to a PVC) from the Disk source list.
5. Browse to the image on your local machine and set the disk size.
6. Click Customize VirtualMachine.
7. Click Create VirtualMachine.

Procedure

1. Upload the Windows image as a new PVC:

   1. Navigate to Storage PersistentVolumeClaims in the web console.
   2. Click Create PersistentVolumeClaim With Data upload form.
   3. Browse to the Windows image and select it.

   4. Enter the PVC name, select the storage class and size and then click Upload.

      The Windows image is uploaded to a PVC.

2. Configure a new VM by cloning the uploaded PVC:

   1. Navigate to Virtualization Catalog.
   2. Select a Windows template tile and click Customize VirtualMachine.
   3. Select Clone (clone PVC) from the Disk source list.
   4. Select the PVC project, the Windows image PVC, and the disk size.

3. Apply the answer file to the VM:

   1. Click Customize VirtualMachine parameters.
   2. On the Sysprep section of the Scripts tab, click Edit.
   3. Browse to the autounattend.xml answer file and click Save.

4. Set the run strategy of the VM:

   1. Clear Start this VirtualMachine after creation so that the VM does not start immediately.
   2. Click Create VirtualMachine.
   3. On the YAML tab, replace running:false with runStrategy: RerunOnFailure and click Save.

5. Click the options menu and select Start.

   The VM boots from the sysprep disk containing the autounattend.xml answer file.

Procedure

1. Upload the image by running the virtctl image-upload command:

   $ virtctl image-upload dv <datavolume_name> \ 1
     --size=<datavolume_size> \ 2
     --image-path=</path/to/image> \ 3

   1

   The name of the data volume.

   2

   The size of the data volume. For example: --size=500Mi, --size=1G

   3

   The file path of the image.

   Note
   • If you do not want to create a new data volume, omit the --size parameter and include the --no-create flag.
   • When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk.
   • To allow insecure server connections when using HTTPS, use the --insecure parameter. When you use the --insecure flag, the authenticity of the upload endpoint is not verified.

2. Optional. To verify that a data volume was created, view all data volumes by running the following command:

   $ oc get dvs

Procedure

1. Navigate to Virtualization VirtualMachines in the web console.
2. Select a VM to open the VirtualMachine details page.
3. Click Actions.
4. Select Clone.
5. On the Clone VirtualMachine page, enter the name of the new VM.
6. (Optional) Select the Start cloned VM checkbox to start the cloned VM.
7. Click Clone.

Procedure

1. Navigate to Virtualization VirtualMachines in the web console.
2. Select a VM to open the VirtualMachine details page.
3. Click the Snapshots tab.
4. Click the actions menu for the snapshot you want to copy.
5. Select Create VirtualMachine.
6. Enter the name of the virtual machine.
7. (Optional) Select the Start this VirtualMachine after creation checkbox to start the new virtual machine.
8. Click Create.

Procedure

1. Navigate to Virtualization Catalog in the web console.
2. Click a template tile without an available boot source.
3. Click Customize VirtualMachine.
4. On the Customize template parameters page, expand Storage and select PVC (clone PVC) from the Disk source list.
5. Enter the image URL. Example: https://access.redhat.com/downloads/content/69/ver=/rhel---7/7.9/x86_64/product-software
6. Enter the container image URL. Example: https://mirror.arizona.edu/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2
7. Select the PVC project and the PVC name.
8. Set the disk size.
9. Click Next.
10. Click Create VirtualMachine.

Procedure

1. On the Virtualization VirtualMachines page, click a VM to open the VirtualMachine details page.
2. Click the Console tab. The VNC console session starts automatically.

3. Optional: To switch to the vGPU display of a Windows VM, select Ctl + Alt + 2 from the Send key list.

   • Select Ctl + Alt + 1 from the Send key list to restore the default display.
4. To end the console session, click outside the console pane and then click Disconnect.

Procedure

1. Run the following command to start the console session:

   $ virtctl vnc <vm_name>

2. If the connection fails, run the following command to collect troubleshooting information:

   $ virtctl vnc <vm_name> -v 4

Procedure

1. Enable the feature gate in the HyperConverged (HCO) custom resource (CR):

   $ oc patch hyperconverged kubevirt-hyperconverged -n openshift-cnv --type json -p '[{"op": "replace", "path": "/spec/featureGates/deployVmConsoleProxy", "value": true}]'

2. Generate a token by entering the following command:

   $ curl --header "Authorization: Bearer ${TOKEN}" \
        "https://api.<cluster_fqdn>/apis/token.kubevirt.io/v1alpha1/namespaces/<namespace>/virtualmachines/<vm_name>/vnc?duration=<duration>"

   The <duration> parameter can be set in hours and minutes, with a minimum duration of 10 minutes. For example: 5h30m. If this parameter is not set, the token is valid for 10 minutes by default.

   Sample output:

   { "token": "eyJhb..." }

3. Optional: Use the token provided in the output to create a variable:

   $ export VNC_TOKEN="<token>"

Verification

1. Log in to the cluster by entering the following command:

   $ oc login --token ${VNC_TOKEN}

2. Test access to the VNC console of the VM by using the virtctl command:

   $ virtctl vnc <vm_name> -n <namespace>

Procedure

1. On the Virtualization VirtualMachines page, click a VM to open the VirtualMachine details page.
2. Click the Console tab. The VNC console session starts automatically.
3. Click Disconnect to end the VNC console session. Otherwise, the VNC console session continues to run in the background.
4. Select Serial console from the console list.
5. To end the console session, click outside the console pane and then click Disconnect.

Procedure

1. Run the following command to start the console session:

   $ virtctl console <vm_name>

2. Press Ctrl+] to end the console session.

Procedure

1. On the Virtualization VirtualMachines page, click a VM to open the VirtualMachine details page.
2. Click the Console tab. The VNC console session starts automatically.
3. Click Disconnect to end the VNC console session. Otherwise, the VNC console session continues to run in the background.
4. Select Desktop viewer from the console list.
5. Click Create RDP Service to open the RDP Service dialog.
6. Select Expose RDP Service and click Save to create a node port service.
7. Click Launch Remote Desktop to download an .rdp file and launch the desktop viewer.

Procedure

1. Navigate to Virtualization VirtualMachines.
2. Click a VM to view the VirtualMachine details page.
3. On the Configuration tab, click the Network interfaces tab.
4. Click Add network interface.
5. Enter the interface name and select the network attachment definition from the Network list.
6. Click Save.
7. Restart the VM to apply the changes.

Procedure

1. Obtain the IP address of the VM by running the following command:

   $ oc describe vm <vm_name> -n <namespace>

   Example output

   # ...
   Interfaces:
     Interface Name:  eth0
     Ip Address:      10.244.0.37/24
     Ip Addresses:
       10.244.0.37/24
       fe80::858:aff:fef4:25/64
     Mac:             0a:58:0a:f4:00:25
     Name:            default
   # ...

2. Connect to the VM by running the following command:

   $ ssh <user_name>@<ip_address> -i <ssh_key>

   Example

   $ ssh cloud-user@10.244.0.37 -i ~/.ssh/id_rsa_cloud-user

Procedure

1. Click Pipelines Pipelines in the side menu.
2. Select a pipeline to open the Pipeline details page.
3. From the Actions list, select Start. The Start Pipeline dialog is displayed.
4. Keep the default values for the parameters and then click Start to run the pipeline. The Details tab tracks the progress of each task and displays the pipeline status.

Procedure

1. To run the Windows 10 installer pipeline, create the following PipelineRun manifest:

   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     generateName: windows10-installer-run-
     labels:
       pipelinerun: windows10-installer-run
   spec:
     params:
     - name: winImageDownloadURL
       value: <link_to_windows_10_iso> 1
     pipelineRef:
       name: windows10-installer
     taskRunSpecs:
       - pipelineTaskName: copy-template
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template
         taskServiceAccountName: modify-vm-template-task
       - pipelineTaskName: create-vm-from-template
         taskServiceAccountName: create-vm-from-template-task
       - pipelineTaskName: wait-for-vmi-status
         taskServiceAccountName: wait-for-vmi-status-task
       - pipelineTaskName: create-base-dv
         taskServiceAccountName: modify-data-object-task
       - pipelineTaskName: cleanup-vm
         taskServiceAccountName: cleanup-vm-task
     status: {}

   1

   Specify the URL for the Windows 10 64-bit ISO file. The product language must be English (United States).

2. Apply the PipelineRun manifest:

   $ oc apply -f windows10-installer-run.yaml

3. To run the Windows 10 customize pipeline, create the following PipelineRun manifest:

   apiVersion: tekton.dev/v1beta1
   kind: PipelineRun
   metadata:
     generateName: windows10-customize-run-
     labels:
       pipelinerun: windows10-customize-run
   spec:
     params:
       - name: allowReplaceGoldenTemplate
         value: true
       - name: allowReplaceCustomizationTemplate
         value: true
     pipelineRef:
       name: windows10-customize
     taskRunSpecs:
       - pipelineTaskName: copy-template-customize
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template-customize
         taskServiceAccountName: modify-vm-template-task
       - pipelineTaskName: create-vm-from-template
         taskServiceAccountName: create-vm-from-template-task
       - pipelineTaskName: wait-for-vmi-status
         taskServiceAccountName: wait-for-vmi-status-task
       - pipelineTaskName: create-base-dv
         taskServiceAccountName: modify-data-object-task
       - pipelineTaskName: cleanup-vm
         taskServiceAccountName: cleanup-vm-task
       - pipelineTaskName: copy-template-golden
         taskServiceAccountName: copy-template-task
       - pipelineTaskName: modify-vm-template-golden
         taskServiceAccountName: modify-vm-template-task
   status: {}

4. Apply the PipelineRun manifest:

   $ oc apply -f windows10-customize-run.yaml

Procedure

1. Set limits for a VM by editing the VirtualMachine manifest. For example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     name: with-limits
   spec:
     running: false
     template:
       spec:
         domain:
   # ...
           resources:
             requests:
               memory: 128Mi
             limits:
               memory: 256Mi  1

   1

   This configuration is supported because the limits.memory value is at least 100Mi larger than the requests.memory value.

2. Save the VirtualMachine manifest.

Procedure

1. From the side menu, click Virtualization Overview.
2. Select the Settings tab.
3. Select the Cluster tab.
4. Expand Resource management.

5. Enable or disable the feature for all nodes:

   • Set Kernel Samepage Merging (KSM) to on.
   • Set Kernel Samepage Merging (KSM) to off.

Procedure

1. Open the HyperConverged CR in your default editor by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

2. Edit the ksmConfiguration stanza:

   • To enable the KSM activation feature for all nodes, set the nodeLabelSelector value to {}. For example:

     apiVersion: hco.kubevirt.io/v1beta1
     kind: HyperConverged
     metadata:
       name: kubevirt-hyperconverged
       namespace: openshift-cnv
     spec:
       configuration:
         ksmConfiguration:
           nodeLabelSelector: {}
     # ...

   • To enable the KSM activation feature on a subset of nodes, edit the nodeLabelSelector field. Add syntax that matches the nodes where you want OpenShift Virtualization to enable KSM. For example, the following configuration allows OpenShift Virtualization to enable KSM on nodes where both <first_example_key> and <second_example_key> are set to "true":

     apiVersion: hco.kubevirt.io/v1beta1
     kind: HyperConverged
     metadata:
       name: kubevirt-hyperconverged
       namespace: openshift-cnv
     spec:
       configuration:
         ksmConfiguration:
           nodeLabelSelector:
             matchLabels:
               <first_example_key>: "true"
               <second_example_key>: "true"
     # ...

   • To disable the KSM activation feature, delete the ksmConfiguration stanza. For example:

     apiVersion: hco.kubevirt.io/v1beta1
     kind: HyperConverged
     metadata:
       name: kubevirt-hyperconverged
       namespace: openshift-cnv
     spec:
       configuration:
     # ...

3. Save the file.

Procedure

1. Open the HyperConverged CR by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

2. Edit the spec.certConfig fields as shown in the following example. To avoid overloading the system, ensure that all values are greater than or equal to 10 minutes. Express all values as strings that comply with the golang ParseDuration format.

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     certConfig:
       ca:
         duration: 48h0m0s
         renewBefore: 24h0m0s 1
       server:
         duration: 24h0m0s  2
         renewBefore: 12h0m0s  3

   1

   The value of ca.renewBefore must be less than or equal to the value of ca.duration.

   2

   The value of server.duration must be less than or equal to the value of ca.duration.

   3

   The value of server.renewBefore must be less than or equal to the value of server.duration.

3. Apply the YAML file to your cluster.

Procedure

1. Open the HyperConverged CR by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

2. Add the defaultCPUModel field to the CR and set the value to the name of a CPU model that exists in the cluster:

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
    name: kubevirt-hyperconverged
    namespace: openshift-cnv
   spec:
     defaultCPUModel: "EPYC"

3. Apply the YAML file to your cluster.

Procedure

1. Edit or create a VirtualMachine manifest file. Use the spec.firmware.bootloader stanza to configure UEFI mode:

   Booting in UEFI mode with secure boot active

   apiversion: kubevirt.io/v1
   kind: VirtualMachine
   metadata:
     labels:
       special: vm-secureboot
     name: vm-secureboot
   spec:
     template:
       metadata:
         labels:
           special: vm-secureboot
       spec:
         domain:
           devices:
             disks:
             - disk:
                 bus: virtio
               name: containerdisk
           features:
             acpi: {}
             smm:
               enabled: true 1
           firmware:
             bootloader:
               efi:
                 secureBoot: true 2
   # ...

   1

   OpenShift Virtualization requires System Management Mode (SMM) to be enabled for Secure Boot in UEFI mode to occur.

   2

   OpenShift Virtualization supports a VM with or without Secure Boot when using UEFI mode. If Secure Boot is enabled, then UEFI mode is required. However, UEFI mode can be enabled without using Secure Boot.

2. Apply the manifest to your cluster by running the following command:

   $ oc create -f <file_name>.yaml

Procedure

1. Configure a PXE network on the cluster:

   1. Create the network attachment definition file for PXE network pxe-net-conf:

      apiVersion: "k8s.cni.cncf.io/v1"
      kind: NetworkAttachmentDefinition
      metadata:
        name: pxe-net-conf 1
      spec:
        config: |
          {
            "cniVersion": "0.3.1",
            "name": "pxe-net-conf", 2
            "type": "bridge", 3
            "bridge": "bridge-interface", 4
            "macspoofchk": false, 5
            "vlan": 100, 6
            "preserveDefaultVlan": false 7
          }

      1

      The name for the NetworkAttachmentDefinition object.

      2

      The name for the configuration. It is recommended to match the configuration name to the name value of the network attachment definition.

      3

      The actual name of the Container Network Interface (CNI) plugin that provides the network for this network attachment definition. This example uses a Linux bridge CNI plugin. You can also use an OVN-Kubernetes localnet or an SR-IOV CNI plugin.

      4

      The name of the Linux bridge configured on the node.

      5

      Optional: A flag to enable the MAC spoof check. When set to true, you cannot change the MAC address of the pod or guest interface. This attribute allows only a single MAC address to exit the pod, which provides security against a MAC spoofing attack.

      6

      Optional: The VLAN tag. No additional VLAN configuration is required on the node network configuration policy.

      7

      Optional: Indicates whether the VM connects to the bridge through the default VLAN. The default value is true.

2. Create the network attachment definition by using the file you created in the previous step:

   $ oc create -f pxe-net-conf.yaml

3. Edit the virtual machine instance configuration file to include the details of the interface and network.

   1. Specify the network and MAC address, if required by the PXE server. If the MAC address is not specified, a value is assigned automatically.

      Ensure that bootOrder is set to 1 so that the interface boots first. In this example, the interface is connected to a network called <pxe-net>:

      interfaces:
      - masquerade: {}
        name: default
      - bridge: {}
        name: pxe-net
        macAddress: de:00:00:00:00:de
        bootOrder: 1

      Note

      Boot order is global for interfaces and disks.

   2. Assign a boot device number to the disk to ensure proper booting after operating system provisioning.

      Set the disk bootOrder value to 2:

      devices:
        disks:
        - disk:
            bus: virtio
          name: containerdisk
          bootOrder: 2

   3. Specify that the network is connected to the previously created network attachment definition. In this scenario, <pxe-net> is connected to the network attachment definition called <pxe-net-conf>:

      networks:
      - name: default
        pod: {}
      - name: pxe-net
        multus:
          networkName: pxe-net-conf

4. Create the virtual machine instance:

   $ oc create -f vmi-pxe-boot.yaml

   Example output

     virtualmachineinstance.kubevirt.io "vmi-pxe-boot" created

5. Wait for the virtual machine instance to run:

   $ oc get vmi vmi-pxe-boot -o yaml | grep -i phase
     phase: Running

6. View the virtual machine instance using VNC:

   $ virtctl vnc vmi-pxe-boot

7. Watch the boot screen to verify that the PXE boot is successful.

8. Log in to the virtual machine instance:

   $ virtctl console vmi-pxe-boot

Verification

1. Verify the interfaces and MAC address on the virtual machine and that the interface connected to the bridge has the specified MAC address. In this case, we used eth1 for the PXE boot, without an IP address. The other interface, eth0, got an IP address from OpenShift Container Platform.

   $ ip addr

   Example output

   ...
   3. eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
      link/ether de:00:00:00:00:de brd ff:ff:ff:ff:ff:ff

Procedure

1. In your virtual machine configuration, add the resources.requests.memory and memory.hugepages.pageSize parameters to the spec.domain. The following configuration snippet is for a virtual machine that requests a total of 4Gi memory with a page size of 1Gi:

   kind: VirtualMachine
   # ...
   spec:
     domain:
       resources:
         requests:
           memory: "4Gi" 1
       memory:
         hugepages:
           pageSize: "1Gi" 2
   # ...

   1

   The total amount of memory requested for the virtual machine. This value must be divisible by the page size.

   2

   The size of each huge page. Valid values for x86_64 architecture are 1Gi and 2Mi. The page size must be smaller than the requested memory.

2. Apply the virtual machine configuration:

   $ oc apply -f <virtual_machine>.yaml

Procedure

1. In the OpenShift Container Platform console, click Virtualization VirtualMachines from the side menu.
2. Select a virtual machine to open the VirtualMachine details page.
3. On the Configuration Scheduling tab, click the edit icon beside Dedicated Resources.
4. Select Schedule this workload with dedicated resources (guaranteed policy).
5. Click Save.

Procedure

1. Log in to the OpenShift Container Platform web console.

2. Create the required namespace for the Kube Descheduler Operator.

   1. Navigate to Administration Namespaces and click Create Namespace.
   2. Enter openshift-kube-descheduler-operator in the Name field, enter openshift.io/cluster-monitoring=true in the Labels field to enable descheduler metrics, and click Create.

3. Install the Kube Descheduler Operator.

   1. Navigate to Operators OperatorHub.
   2. Type Kube Descheduler Operator into the filter box.
   3. Select the Kube Descheduler Operator and click Install.
   4. On the Install Operator page, select A specific namespace on the cluster. Select openshift-kube-descheduler-operator from the drop-down menu.
   5. Adjust the values for the Update Channel and Approval Strategy to the desired values.
   6. Click Install.

4. Create a descheduler instance.

   1. From the Operators Installed Operators page, click the Kube Descheduler Operator.
   2. Select the Kube Descheduler tab and click Create KubeDescheduler.

   3. Edit the settings as necessary.

      1. To evict pods instead of simulating the evictions, change the Mode field to Automatic.

      2. Expand the Profiles section and select DevPreviewLongLifecycle. The AffinityAndTaints profile is enabled by default.

         Important

         The only profile currently available for OpenShift Virtualization is DevPreviewLongLifecycle.

Procedure

1. Before starting the VM, add the descheduler.alpha.kubernetes.io/evict annotation to the VirtualMachine CR:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachine
   spec:
     template:
       metadata:
         annotations:
           descheduler.alpha.kubernetes.io/evict: "true"

2. If you did not already set the DevPreviewLongLifecycle profile in the web console during installation, specify the DevPreviewLongLifecycle in the spec.profile section of the KubeDescheduler object:

   apiVersion: operator.openshift.io/v1
   kind: KubeDescheduler
   metadata:
     name: cluster
     namespace: openshift-kube-descheduler-operator
   spec:
     deschedulingIntervalSeconds: 3600
     profiles:
     - DevPreviewLongLifecycle
     mode: Predictive 1

   1

   By default, the descheduler does not evict pods. To evict pods, set mode to Automatic.