Chapter 8. Live migration
8.1. Virtual machine live migration
8.1.1. Prerequisites
- Before using LiveMigration, ensure that the storage class used by the virtual machine has a PersistentVolumeClaim (PVC) with a shared ReadWriteMany (RWX) access mode. See Storage defaults for DataVolumes to ensure your storage settings are correct.
8.1.2. Understanding live migration
Live migration is the process of moving a running virtual machine instance to another node in the cluster without interrupting the virtual workload or access. This can be a manual process, if you select virtual machine instance to migrate to another node, or an automatic process, if the virtual machine instance has a LiveMigrate eviction strategy and the node on which it is running is placed into maintenance.
Virtual machines must have a PersistentVolumeClaim (PVC) with a shared ReadWriteMany (RWX) access mode to be live migrated.
8.1.3. Updating access mode for LiveMigration
For LiveMigration to function properly, you must use the ReadWriteMany (RWX) access mode. Use this procedure to update the access mode, if needed.
Procedure
To set the RWX access mode, run the following
oc patchcommand:$ oc patch -n openshift-cnv \ cm kubevirt-storage-class-defaults \ -p '{"data":{"'$<STORAGE_CLASS>'.accessMode":"ReadWriteMany"}}'
Additional resources:
8.2. Live migration limits and timeouts
Live migration limits and timeouts are applied so that migration processes do not overwhelm the cluster. Configure these settings by editing the kubevirt-config configuration file.
8.2.1. Configuring live migration limits and timeouts
Configure live migration limits and timeouts for the cluster by adding updated key:value fields to the kubevirt-config configuration file, which is located in the openshift-cnv namespace.
Procedure
Edit the
kubevirt-configconfiguration file and add the necessary live migration parameters. The following example shows the default values:$ oc edit configmap kubevirt-config -n openshift-cnv
apiVersion: v1 kind: ConfigMap metadata: name: kubevirt-config namespace: kubevirt labels: kubevirt.io: "" data: feature-gates: "LiveMigration" migrations: |- parallelMigrationsPerCluster: 5 parallelOutboundMigrationsPerNode: 2 bandwidthPerMigration: 64Mi completionTimeoutPerGiB: 800 progressTimeout: 150
8.2.2. Cluster-wide live migration limits and timeouts
| Parameter | Description | Default |
|---|---|---|
|
| Number of migrations running in parallel in the cluster. | 5 |
|
| Maximum number of outbound migrations per node. | 2 |
|
| Bandwidth limit of each migration, in MiB/s. | 64Mi |
|
|
The migration will be canceled if it has not completed in this time, in seconds per GiB of memory. For example, a virtual machine instance with 6GiB memory will timeout if it has not completed migration in 4800 seconds. If the | 800 |
|
| The migration will be canceled if memory copy fails to make progress in this time, in seconds. | 150 |
8.3. Migrating a virtual machine instance to another node
Manually initiate a live migration of a virtual machine instance to another node using either the web console or the CLI.
8.3.1. Initiating live migration of a virtual machine instance in the web console
Migrate a running virtual machine instance to a different node in the cluster.
The Migrate Virtual Machine action is visible to all users but only admin users can initiate a virtual machine migration.
Procedure
-
In the container-native virtualization console, click Workloads
Virtual Machines. You can initiate the migration from this screen, which makes it easier to perform actions on multiple virtual machines in the one screen, or from the Virtual Machine Details screen where you can view comprehensive details of the selected virtual machine:
-
Click the Options menu
at the end of virtual machine and select Migrate Virtual Machine.
-
Click the virtual machine name to open the Virtual Machine Details screen and click Actions
Migrate Virtual Machine.
-
Click the Options menu
- Click Migrate to migrate the virtual machine to another node.
8.3.2. Initiating live migration of a virtual machine instance in the CLI
Initiate a live migration of a running virtual machine instance by creating a VirtualMachineInstanceMigration object in the cluster and referencing the name of the virtual machine instance.
Procedure
Create a
VirtualMachineInstanceMigrationconfiguration file for the virtual machine instance to migrate. For example,vmi-migrate.yaml:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachineInstanceMigration metadata: name: migration-job spec: vmiName: vmi-fedora
Create the object in the cluster:
$ oc create -f vmi-migrate.yaml
The VirtualMachineInstanceMigration object triggers a live migration of the virtual machine instance. This object exists in the cluster for as long as the virtual machine instance is running, unless manually deleted.
8.4. Monitoring live migration of a virtual machine instance
You can monitor the progress of a live migration of a virtual machine instance from either the web console or the CLI.
8.4.1. Monitoring live migration of a virtual machine instance in the web console
For the duration of the migration, the virtual machine has a status of Migrating. This status is displayed in the Virtual Machines list or in the Virtual Machine Details screen for the migrating virtual machine.
Procedure
-
In the container-native virtualization console, click Workloads
Virtual Machines.
8.4.2. Monitoring live migration of a virtual machine instance in the CLI
The status of the virtual machine migration is stored in the Status component of the VirtualMachineInstance configuration.
Procedure
Use the
oc describecommand on the migrating virtual machine instance:$ oc describe vmi vmi-fedora
... Status: Conditions: Last Probe Time: <nil> Last Transition Time: <nil> Status: True Type: LiveMigratable Migration Method: LiveMigration Migration State: Completed: true End Timestamp: 2018-12-24T06:19:42Z Migration UID: d78c8962-0743-11e9-a540-fa163e0c69f1 Source Node: node2.example.com Start Timestamp: 2018-12-24T06:19:35Z Target Node: node1.example.com Target Node Address: 10.9.0.18:43891 Target Node Domain Detected: true
8.5. Cancelling the live migration of a virtual machine instance
Cancel the live migration so that the virtual machine instance remains on the original node.
You can cancel a live migration from either the web console or the CLI.
8.5.1. Cancelling live migration of a virtual machine instance in the web console
A live migration of the virtual machine instance can be cancelled using the Options menu
found on each virtual machine in the Workloads
Procedure
-
In the container-native virtualization console, click Workloads
Virtual Machines. You can cancel the migration from this screen, which makes it easier to perform actions on multiple virtual machines in the one screen, or from the Virtual Machine Details screen where you can view comprehensive details of the selected virtual machine:
-
Click the Options menu
at the end of virtual machine and select Cancel Virtual Machine Migration.
-
Click the virtual machine name to open the Virtual Machine Details screen and click Actions
Cancel Virtual Machine Migration.
-
Click the Options menu
- Click Cancel Migration to cancel the virtual machine live migration.
8.5.2. Cancelling live migration of a virtual machine instance in the CLI
Cancel the live migration of a virtual machine instance by deleting the VirtualMachineInstanceMigration object associated with the migration.
Procedure
Delete the
VirtualMachineInstanceMigrationobject that triggered the live migration,migration-jobin this example:$ oc delete vmim migration-job
8.6. Configuring virtual machine eviction strategy
The LiveMigrate eviction strategy ensures that a virtual machine instance is not interrupted if the node is placed into maintenance or drained. Virtual machines instances with this eviction strategy will be live migrated to another node.
8.6.1. Configuring custom virtual machines with the LiveMigration eviction strategy
You only need to configure the LiveMigration eviction strategy on custom virtual machines. Common templates have this eviction strategy configured by default.
Procedure
Add the
evictionStrategy: LiveMigrateoption to thespecsection in the virtual machine configuration file. This example usesoc editto update the relevant snippet of theVirtualMachineconfiguration file:$ oc edit vm <custom-vm> -n <my-namespace>
apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: custom-vm spec: terminationGracePeriodSeconds: 30 evictionStrategy: LiveMigrate domain: resources: requests: ...Restart the virtual machine for the update to take effect:
$ virtctl restart <custom-vm> -n <my-namespace>
