Chapter 13. Live migration

Procedure

1. Before updating to OpenShift Virtualization 4.20, create a temporary ClusterRole object. For example:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     labels:
       rbac.authorization.k8s.io/aggregate-to-admin=true 

   1

   
     name: kubevirt.io:upgrademigrate
   rules:
   - apiGroups:
     - subresources.kubevirt.io
     resources:
     - virtualmachines/migrate
     verbs:
     - update
   - apiGroups:
     - kubevirt.io
     resources:
     - virtualmachineinstancemigrations
     verbs:
     - get
     - delete
     - create
     - update
     - patch
     - list
     - watch
     - deletecollection

   Copy to Clipboard Toggle word wrap

   1

   This cluster role is aggregated into the admin role before you update OpenShift Virtualization. The update process does not modify it, ensuring the previous behavior is maintained.

2. Add the cluster role manifest to the cluster by running the following command:

   $ oc apply -f <cluster_role_file_name>.yaml

   Copy to Clipboard Toggle word wrap
3. Update OpenShift Virtualization to version 4.20.

4. Bind the kubevirt.io:migrate cluster role to trusted users or groups by running one of the following commands, replacing <namespace>, <first_user>, <second_user>, and <group_name> with your own values.

   • To bind the role at the namespace level, run the following command:

     $ oc create -n <namespace> rolebinding kvmigrate --clusterrole=kubevirt.io:migrate --user=<first_user> --user=<second_user> --group=<group_name>

     Copy to Clipboard Toggle word wrap

   • To bind the role at the cluster level, run the following command:

     $ oc create clusterrolebinding kvmigrate --clusterrole=kubevirt.io:migrate --user=<first_user> --user=<second_user> --group=<group_name>

     Copy to Clipboard Toggle word wrap

5. When you have bound the kubevirt.io:migrate role to all necessary users, delete the temporary ClusterRole object by running the following command:

   $ oc delete clusterrole kubevirt.io:upgrademigrate

   Copy to Clipboard Toggle word wrap

   After you delete the temporary cluster role, only users with the kubevirt.io:migrate role can create, delete, and update live migration requests.

Procedure

1. Edit the HyperConverged CR and add the necessary parameters for migrating heavy workloads:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

   Copy to Clipboard Toggle word wrap

   Example configuration file

   apiVersion: hco.kubevirt.io/v1beta1
   kind: HyperConverged
   metadata:
     name: kubevirt-hyperconverged
     namespace: openshift-cnv
   spec:
     liveMigrationConfig:
       bandwidthPerMigration: 0Mi 

   1

   
       completionTimeoutPerGiB: 150 

   2

   
       parallelMigrationsPerCluster: 5 

   3

   
       parallelOutboundMigrationsPerNode: 1 

   4

   
       progressTimeout: 150 

   5

   
       allowPostCopy: true 

   6

   Copy to Clipboard Toggle word wrap

   1

   Bandwidth limit of each migration, where the value is the quantity of bytes per second. The default is 0, which is unlimited.

   2

   The migration is canceled if it is not completed in this time, and triggers post copy mode, when post copy is enabled. This value is measured in seconds per GiB of memory. You can lower completionTimeoutPerGiB to trigger post copy mode earlier in the migration process, or raise the completionTimeoutPerGiB to trigger post copy mode later in the migration process.

   3

   Number of migrations running in parallel in the cluster. The default is 5. Keeping the parallelMigrationsPerCluster setting low is better when migrating heavy workloads.

   4

   Maximum number of outbound migrations per node. Configure a single VM per node for heavy workloads.

   5

   The migration is canceled if memory copy fails to make progress in this time. This value is measured in seconds. Increase this parameter for large memory sizes running heavy workloads.

   6

   Use post copy mode when memory dirty rates are high to ensure the migration converges. Set allowPostCopy to true to enable post copy mode.

2. Optional: If your main network is too busy for the migration, configure a secondary, dedicated migration network.

Procedure

1. Create a migration manifest YAML file. For example:

   apiVersion: kubevirt.io/v1
   kind: VirtualMachineInstanceMigration
   metadata:
     name: migration-job
   spec:
     vmiName: vmi-fedora
     addedNodeSelector:
       accelerator: gpu-enabled23
       kubernetes.io/hostname: "ip-172-28-114-199.example"

   Copy to Clipboard Toggle word wrap

   where:

   vmiName

   Specifies the name of the running VM (for example, vmi-fedora).

   addedNodeSelector

   Specifies additional constraints for selecting the target node.

2. Apply the manifest to the cluster by running the following command:

   $ oc apply -f <file_name>.yaml

   Copy to Clipboard Toggle word wrap

   If no nodes satisfy the constraints, the migration is declared a failure after a timeout. The VM remains unaffected.