7.15. Virtual machine disks

1. PVCs must request a ReadWriteMany access mode.
2. Storage provider must support both Kubernetes and CSI snapshot APIs

Procedure

1. Create the MachineConfig file. For example:

   $ touch machineconfig.yaml

2. Edit the file, ensuring that you include the directory where you want the hostpath provisioner to create PVs. For example:

   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     name: 50-set-selinux-for-hostpath-provisioner
     labels:
       machineconfiguration.openshift.io/role: worker
   spec:
     config:
       ignition:
         version: 2.2.0
       systemd:
         units:
           - contents: |
               [Unit]
               Description=Set SELinux chcon for hostpath provisioner
               Before=kubelet.service
   
               [Service]
               ExecStart=/usr/bin/chcon -Rt container_file_t <path/to/backing/directory> 1
   
               [Install]
               WantedBy=multi-user.target
             enabled: true
             name: hostpath-provisioner.service

   1

   Specify the backing directory where you want the provisioner to create PVs.

3. Create the MachineConfig object:

   $ oc create -f machineconfig.yaml -n <namespace>

Procedure

1. Create the HostPathProvisioner custom resource file. For example:

   $ touch hostpathprovisioner_cr.yaml

2. Edit the file, ensuring that the spec.pathConfig.path value is the directory where you want the hostpath provisioner to create PVs. For example:

   apiVersion: hostpathprovisioner.kubevirt.io/v1alpha1
   kind: HostPathProvisioner
   metadata:
     name: hostpath-provisioner
   spec:
     imagePullPolicy: IfNotPresent
     pathConfig:
       path: "</path/to/backing/directory>" 1
       useNamingPrefix: "false" 2

   1

   Specify the backing directory where you want the provisioner to create PVs.

   2

   Change this value to true if you want to use the name of the PersistentVolumeClaim (PVC) that is bound to the created PV as the prefix of the directory name.

   注意

   If you did not create the backing directory, the provisioner attempts to create it for you. If you did not apply the container_file_t SELinux context, this can cause Permission denied errors.

3. Create the custom resource in the openshift-cnv namespace:

   $ oc create -f hostpathprovisioner_cr.yaml -n openshift-cnv

Procedure

1. Create a YAML file for defining the storage class. For example:

   $ touch storageclass.yaml

2. Edit the file. For example:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: hostpath-provisioner 1
   provisioner: kubevirt.io/hostpath-provisioner
   reclaimPolicy: Delete 2
   volumeBindingMode: WaitForFirstConsumer 3

   1

   You can optionally rename the storage class by changing this value.

   2

   The two possible reclaimPolicy values are Delete and Retain. If you do not specify a value, the storage class defaults to Delete.

   3

   The volumeBindingMode value determines when dynamic provisioning and volume binding occur. Specify WaitForFirstConsumer to delay the binding and provisioning of a PV until after a Pod that uses the PersistentVolumeClaim (PVC) is created. This ensures that the PV meets the Pod’s scheduling requirements.

1. Create the StorageClass object:

   $ oc create -f storageclass.yaml

Procedure

1. Edit the cdiconfig/config by running the following command:

   $ oc edit cdiconfig/config

2. Change the default CPU and memory requests and limits by editing the spec: podResourceRequirements property of the CDIConfig object:

   apiVersion: cdi.kubevirt.io/v1alpha1
   kind: CDIConfig
   metadata:
     labels:
       app: containerized-data-importer
       cdi.kubevirt.io: ""
     name: config
   spec:
       podResourceRequirements:
       limits:
         cpu: "4"
         memory: "1Gi"
       requests:
         cpu: "1"
         memory: "250Mi"
   ...

3. Save and exit the editor to update the CDIConfig object.

Procedure

1. Create a DataVolume configuration that specifies spec: source: upload{}:

   apiVersion: cdi.kubevirt.io/v1alpha1
   kind: DataVolume
   metadata:
     name: <upload-datavolume> 1
   spec:
     source:
         upload: {}
     pvc:
       accessModes:
         - ReadWriteOnce
       resources:
         requests:
           storage: <2Gi> 2

   1

   The name of the DataVolume.

   2

   The size of the DataVolume. Ensure that this value is greater than or equal to the size of the disk that you upload.

2. Create the DataVolume by running the following command:

   $ oc create -f <upload-datavolume>.yaml

Procedure

1. Identify the following items:

   • The name of the upload DataVolume that you want to use. If this DataVolume does not exist, it is created automatically.
   • The size of the DataVolume, if you want it to be created during the upload procedure. The size must be greater than or equal to the size of the disk image.
   • The file location of the virtual machine disk image that you want to upload.

2. Upload the disk image by running the virtctl image-upload command. Specify the parameters that you identified in the previous step. For example:

   $ virtctl image-upload dv <datavolume_name> \ 1
   --size=<datavolume_size> \ 2
   --image-path=</path/to/image> \ 3

   1

   The name of the DataVolume.

   2

   The size of the DataVolume. For example: --size=500Mi, --size=1G

   3

   The file path of the virtual machine disk image.

   注意
   • If you do not want to create a new DataVolume, omit the --size parameter and include the --no-create flag.
   • When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk.
   • To allow insecure server connections when using HTTPS, use the --insecure parameter. Be aware that when you use the --insecure flag, the authenticity of the upload endpoint is not verified.

3. Optional. To verify that a DataVolume was created, view all DataVolume objects by running the following command:

   $ oc get dvs

Procedure

1. Log in as root to the node on which to create the local PV. This procedure uses node01 for its examples.

2. Create a file and populate it with null characters so that it can be used as a block device. The following example creates a file loop10 with a size of 2Gb (20 100Mb blocks):

   $ dd if=/dev/zero of=<loop10> bs=100M count=20

3. Mount the loop10 file as a loop device.

   $ losetup </dev/loop10>d3 <loop10> 1 2

   1

   File path where the loop device is mounted.

   2

   The file created in the previous step to be mounted as the loop device.

4. Create a PersistentVolume configuration that references the mounted loop device.

   kind: PersistentVolume
   apiVersion: v1
   metadata:
     name: <local-block-pv10>
     annotations:
   spec:
     local:
       path: </dev/loop10> 1
     capacity:
       storage: <2Gi>
     volumeMode: Block 2
     storageClassName: local 3
     accessModes:
       - ReadWriteOnce
     persistentVolumeReclaimPolicy: Delete
     nodeAffinity:
       required:
         nodeSelectorTerms:
         - matchExpressions:
           - key: kubernetes.io/hostname
             operator: In
             values:
             - <node01> 4

   1

   The path of the loop device on the node.

   2

   Specifies it is a block PV.

   3

   Optional: Set a StorageClass for the PV. If you omit it, the cluster default is used.

   4

   The node on which the block device was mounted.

5. Create the block PV.

   # oc create -f <local-block-pv10.yaml>1

   1

   The filename of the PersistentVolume created in the previous step.

Procedure

1. Create a DataVolume configuration that specifies spec: source: upload{}:

   apiVersion: cdi.kubevirt.io/v1alpha1
   kind: DataVolume
   metadata:
     name: <upload-datavolume> 1
   spec:
     source:
         upload: {}
     pvc:
       accessModes:
         - ReadWriteOnce
       resources:
         requests:
           storage: <2Gi> 2

   1

   The name of the DataVolume.

   2

   The size of the DataVolume. Ensure that this value is greater than or equal to the size of the disk that you upload.

2. Create the DataVolume by running the following command:

   $ oc create -f <upload-datavolume>.yaml

Procedure

1. Identify the following items:

   • The name of the upload DataVolume that you want to use. If this DataVolume does not exist, it is created automatically.
   • The size of the DataVolume, if you want it to be created during the upload procedure. The size must be greater than or equal to the size of the disk image.
   • The file location of the virtual machine disk image that you want to upload.

2. Upload the disk image by running the virtctl image-upload command. Specify the parameters that you identified in the previous step. For example:

   $ virtctl image-upload dv <datavolume_name> \ 1
   --size=<datavolume_size> \ 2
   --image-path=</path/to/image> \ 3

   1

   The name of the DataVolume.

   2

   The size of the DataVolume. For example: --size=500Mi, --size=1G

   3

   The file path of the virtual machine disk image.

   注意
   • If you do not want to create a new DataVolume, omit the --size parameter and include the --no-create flag.
   • When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk.
   • To allow insecure server connections when using HTTPS, use the --insecure parameter. Be aware that when you use the --insecure flag, the authenticity of the upload endpoint is not verified.

3. Optional. To verify that a DataVolume was created, view all DataVolume objects by running the following command:

   $ oc get dvs

Procedure

1. Either create a new local PV on the node, or identify a local PV already on the node:

   • Create a local PV that includes the nodeAffinity.nodeSelectorTerms parameters. The following manifest creates a 10Gi local PV on node01.

     kind: PersistentVolume
     apiVersion: v1
     metadata:
       name: <destination-pv> 1
       annotations:
     spec:
       accessModes:
       - ReadWriteOnce
       capacity:
         storage: 10Gi 2
       local:
         path: /mnt/local-storage/local/disk1 3
       nodeAffinity:
         required:
           nodeSelectorTerms:
           - matchExpressions:
             - key: kubernetes.io/hostname
               operator: In
               values:
               - node01 4
       persistentVolumeReclaimPolicy: Delete
       storageClassName: local
       volumeMode: Filesystem

     1

     The name of the PV.

     2

     The size of the PV. You must allocate enough space, or the cloning operation fails. The size must be the same as or larger than the source PVC.

     3

     The mount path on the node.

     4

     The name of the node where you want to create the PV.

   • Identify a PV that already exists on the target node. You can identify the node where a PV is provisioned by viewing the nodeAffinity field in its configuration:

     $ oc get pv <destination-pv> -o yaml

     The following snippet shows that the PV is on node01:

     Example output

     ...
     spec:
       nodeAffinity:
         required:
           nodeSelectorTerms:
           - matchExpressions:
             - key: kubernetes.io/hostname 1
               operator: In
               values:
               - node01 2
     ...

     1

     The kubernetes.io/hostname key uses the node host name to select a node.

     2

     The host name of the node.

2. Add a unique label to the PV:

   $ oc label pv <destination-pv> node=node01

3. Create a DataVolume manifest that references the following:

   • The PVC name and namespace of the virtual machine.
   • The label you applied to the PV in the previous step.

   • The size of the destination PV.

     apiVersion: cdi.kubevirt.io/v1alpha1
     kind: DataVolume
     metadata:
       name: <clone-datavolume> 1
     spec:
       source:
         pvc:
           name: "<source-vm-disk>" 2
           namespace: "<source-namespace>" 3
       pvc:
         accessModes:
           - ReadWriteOnce
         selector:
           matchLabels:
             node: node01 4
         resources:
           requests:
             storage: <10Gi> 5

     1

     The name of the new DataVolume.

     2

     The name of the source PVC. If you do not know the PVC name, you can find it in the virtual machine configuration: spec.volumes.persistentVolumeClaim.claimName.

     3

     The namespace where the source PVC exists.

     4

     The label that you applied to the PV in the previous step.

     5

     The size of the destination PV.

4. Start the cloning operation by applying the DataVolume manifest to your cluster:

   $ oc apply -f <clone-datavolume.yaml>

Procedure

1. Edit the DataVolume configuration file:

   apiVersion: cdi.kubevirt.io/v1alpha1
   kind: DataVolume
   metadata:
     name: blank-image-datavolume
   spec:
     source:
         blank: {}
     pvc:
       # Optional: Set the storage class or omit to accept the default
       # storageClassName: "hostpath"
       accessModes:
         - ReadWriteOnce
       resources:
         requests:
           storage: 500Mi

2. Create the blank disk image by running the following command:

   $ oc create -f <blank-image-datavolume>.yaml

Procedure

1. Click Workloads Config Maps from the side menu.
2. In the Project list, select openshift-cnv.
3. Click kubevirt-storage-class-defaults to open the Config Map Overview.
4. Click the YAML tab to display the editable configuration.

5. Update the data values with the storage configuration that is appropriate for your underlying storage:

   ...
   data:
     accessMode: ReadWriteOnce 1
     volumeMode: Filesystem 2
     <new>.accessMode: ReadWriteMany 3
     <new>.volumeMode: Block 4

   1

   The default accessMode is ReadWriteOnce.

   2

   The default volumeMode is Filesystem.

   3

   If you add an access mode for a storage class, replace the <new> part of the parameter with the storage class name.

   4

   If you add a volume mode for a storage class, replace the <new> part of the parameter with the storage class name.

6. Click Save to update the config map.

Procedure

1. Edit the ConfigMap by running the following command:

   $ oc edit configmap kubevirt-storage-class-defaults -n openshift-cnv

2. Update the data values of the ConfigMap:

   ...
   data:
     accessMode: ReadWriteOnce 1
     volumeMode: Filesystem 2
     <new>.accessMode: ReadWriteMany 3
     <new>.volumeMode: Block 4

   1

   The default accessMode is ReadWriteOnce.

   2

   The default volumeMode is Filesystem.

   3

   If you add an access mode for storage class, replace the <new> part of the parameter with the storage class name.

   4

   If you add a volume mode for a storage class, replace the <new> part of the parameter with the storage class name.

3. Save and exit the editor to update the config map.

Procedure

1. Create a Dockerfile to build the virtual machine image into a container image. The virtual machine image must be owned by QEMU, which has a UID of 107, and placed in the /disk/ directory inside the container. Permissions for the /disk/ directory must then be set to 0440.

   The following example uses the Red Hat Universal Base Image (UBI) to handle these configuration changes in the first stage, and uses the minimal scratch image in the second stage to store the result:

   $ cat > Dockerfile << EOF
   FROM registry.access.redhat.com/ubi8/ubi:latest AS builder
   ADD --chown=107:107 <vm_image>.qcow2 /disk/ 1
   RUN chmod 0440 /disk/*
   
   FROM scratch
   COPY --from=builder /disk/* /disk/
   EOF

   1

   Where <vm_image> is the virtual machine image in either QCOW2 or RAW format.
   To use a remote virtual machine image, replace <vm_image>.qcow2 with the complete url for the remote image.

2. Build and tag the container:

   $ podman build -t <registry>/<container_disk_name>:latest .

3. Push the container image to the registry:

   $ podman push <registry>/<container_disk_name>:latest

Procedure

1. Ensure that the reclaim policy of the PV is set to Retain:

   1. Check the reclaim policy of the PV:

      $ oc get pv <pv_name> -o yaml | grep 'persistentVolumeReclaimPolicy'

   2. If the persistentVolumeReclaimPolicy is not set to Retain, edit the reclaim policy with the following command:

      $ oc patch pv <pv_name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

2. Ensure that no resources are using the PV:

   $ oc describe pvc <pvc_name> | grep 'Mounted By:'

   Remove any resources that use the PVC before continuing.

3. Delete the PVC to release the PV:

   $ oc delete pvc <pvc_name>

4. Optional: Export the PV configuration to a YAML file. If you manually remove the shared storage later in this procedure, you can refer to this configuration. You can also use spec parameters in this file as the basis to create a new PV with the same storage configuration after you reclaim the PV:

   $ oc get pv <pv_name> -o yaml > <file_name>.yaml

5. Delete the PV:

   $ oc delete pv <pv_name>

6. Optional: Depending on the storage type, you might need to remove the contents of the shared storage folder:

   $ rm -rf <path_to_share_storage>

7. Optional: Create a PV that uses the same storage configuration as the deleted PV. If you exported the reclaimed PV configuration earlier, you can use the spec parameters of that file as the basis for a new PV manifest:

   注意

   To avoid possible conflict, it is good practice to give the new PV object a different name than the one that you deleted.

   $ oc create -f <new_pv_name>.yaml