SupportConsoleDevelopersStart a trialContact

Chapter 9. Advanced migration options

Perform advanced migration operations, such as changing precopy snapshot intervals for warm migration, configuring AIO buffering for performance optimization, creating custom rules for validation, or adding hooks to your migration plan. For performance test data and optimization recommendations, see the "Additional resources" section.

9.2. Changing precopy intervals for warm migration
Copy link

You can change the snapshot interval by patching the ForkliftController custom resource (CR).

Procedure

  • Patch the ForkliftController CR: 

    $ oc patch forkliftcontroller/<forklift-controller> -n openshift-mtv -p '{"spec": {"controller_precopy_interval": <interval_in_minutes>}}' --type=merge

    where:

    <interval_in_minutes>`

    Specifies the precopy interval in minutes. The default value is 60.

    You do not need to restart the forklift-controller pod.

9.3. About AIO buffering for cold migrations
Copy link

You can configure Asynchronous Input/Output (AIO) buffering to optimize Network Block Device (NBD) transport performance for cold migrations in specific scenarios.

AIO buffering adjusts the buffer size and count parameters for NBD transport network file copy (NFC) operations. This optimization is most effective when migrating multiple virtual machines (VMs) concurrently from a single ESXi host.

Important

AIO buffering is only suitable for cold migration use cases. You must remove AIO buffer configuration before performing warm migrations.

9.3.1. Enabling and configuring AIO buffering
Copy link

You can enable and configure Asynchronous Input/Output (AIO) buffering for use with the Migration Toolkit for Virtualization (MTV).

Procedure

  1. Ensure that the forklift-controller pod in the openshift-mtv namespace supports the AIO buffer values. Since the pod name prefix is dynamic, check the pod name by running the following command: 

    oc get pods -n openshift-mtv | grep forklift-controller | awk '{print $1}'

    For example, the output if the pod name prefix is "forklift-controller-667f57c8f8-qllnx" would be: 

    forklift-controller-667f57c8f8-qllnx

  2. Check the environment variables of the pod by running the following command: 

    oc get pod forklift-controller-667f57c8f8-qllnx -n openshift-mtv -o yaml

  3. Check for the following lines in the output: 

    ...
\- name: VIRT\_V2V\_EXTRA\_ARGS
\- name: VIRT\_V2V\_EXTRA\_CONF\_CONFIG\_MAP
...

  4. In the openshift-mtv namespace, edit the ForkliftController custom resource (CR) by performing the following steps:

    1. Access the ForkliftController CR for editing by running the following command: 

      oc edit forkliftcontroller -n openshift-mtv

    2. Add the following lines to the spec section of the ForkliftController CR: 

      virt_v2v_extra_args: "--vddk-config /mnt/extra-v2v-conf/input.conf"
virt_v2v_extra_conf_config_map: "perf"

  5. Create the required config map perf by running the following command: 

    oc -n openshift-mtv create cm perf

  6. Convert the desired buffer configuration values to Base64. For example, for 16/4, run the following command: 

    echo -e "VixDiskLib.nfcAio.Session.BufSizeIn64KB=16\nvixDiskLib.nfcAio.Session.BufCount=4" | base64

    The output will be similar to the following: 

    Vml4RGlza0xpYi5uZmNBaW8uU2Vzc2lvbi5CdWZTaXplSW42NEtCPTE2CnZpeERpc2tMaWIubmZjQWlvLlNlc3Npb24uQnVmQ291bnQ9NAo=

  7. In the config map perf, enter the Base64 string in the binaryData section, for example: 

    apiVersion: v1
kind: ConfigMap
binaryData:
  input.conf: Vml4RGlza0xpYi5uZmNBaW8uU2Vzc2lvbi5CdWZTaXplSW42NEtCPTE2CnZpeERpc2tMaWIubmZjQWlvLlNlc3Npb24uQnVmQ291bnQ9NAo=
metadata:
  name: perf
  namespace: openshift-mtv
  8. Restart the forklift-controller pod to apply the new configuration.
  9. Ensure the VIRT_V2V_EXTRA_ARGS environment variable reflects the updated settings.

  10. Run a migration plan and check the logs of the migration pod. Confirm that the AIO buffer settings are passed as parameters, particularly the --vddk-config value.

    For example, if you run the following command: 

    exec: /usr/bin/virt-v2v … --vddk-config /mnt/extra-v2v-conf/input.conf

    The logs include a section similar to the following, if debug_level = 4: 

    Buffer size calc for 16 value:
(16 * 64 * 1024 = 1048576)
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAio_OpenSession:
Opening an AIO session.
nbdkit: vddk[1]: debug: [NFC INFO] NfcAioInitSession:
Disabling
read-ahead buffer since the AIO buffer size of 1048576 is >=
the read-ahead buffer size of 65536. Explicitly setting flag
'`NFC_AIO_SESSION_NO_NET_READ_AHEAD`'
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAioInitSession: AIO Buffer Size is 1048576
nbdkit: vddk[1]: debug: [NFC VERBOSE] NfcAioInitSession: AIO Buffer
Count is 4

  11. Verify that the correct config map values are in the migration pod. Do this by logging into the migration pod and running the following command: 

    cat /mnt/extra-v2v-conf/input.conf

    Example output is as follows: 

    VixDiskLib.nfcAio.Session.BufSizeIn64KB=16
vixDiskLib.nfcAio.Session.BufCount=4

  12. Optional: Enable debug logs by running the following command. The command converts the configuration to Base64, including a high log level: 

    echo -e
"`VixDiskLib.nfcAio.Session.BufSizeIn64KB=16\nVixDiskLib.nfcAio.Session.BufCount=4\nVixDiskLib.nfc.LogLevel=4`"
| base64
    Note

    Adding a high log level reduces performance and is for debugging purposes only.

9.3.2. Removing AIO buffer configuration
Copy link

You can remove AIO buffer configuration for a cold migration using Migration Toolkit for Virtualization (MTV). You must remove AIO buffer configuration before performing warm migrations using MTV.

Note

The procedure that follows assumes the AIO buffering was enabled and configured according to the procedure in Enabling and configuring AIO buffering.

Procedure

  1. In the openshift-mtv namespace, edit the ForkliftController custom resource (CR) by performing the following steps:

    1. Access the ForkliftController CR for editing by running the following command: 

      oc edit forkliftcontroller -n openshift-mtv

    2. Remove the following lines from the spec section of the ForkliftController CR: 

      virt_v2v_extra_args: "`–vddk-config /mnt/extra-v2v-conf/input.conf`"
virt_v2v_extra_conf_config_map: "`perf`"

  2. Delete the config map named perf: 

    oc delete cm perf -n openshift-mtv
  3. Optional: Restart the forklift-controller pod to ensure that the changes took effect.

The Validation service uses Open Policy Agent (OPA) policy rules to check the suitability of each virtual machine (VM) for migration. The Validation service generates a list of concerns for each VM, which are stored in the Provider Inventory service as VM attributes. The web console displays the concerns for each VM in the provider inventory.

You can create custom rules to extend the default ruleset of the Validation service. For example, you can create a rule that checks whether a VM has multiple disks.

9.4.1. About Rego files
Copy link

Validation rules are written in Rego, the Open Policy Agent (OPA) native query language. The rules are stored as .rego files in the /usr/share/opa/policies/io/konveyor/forklift/_<provider>_ directory of the Validation pod.

Each validation rule is defined in a separate .rego file and tests for a specific condition. If the condition evaluates as true, the rule adds a hash containing the category, label, and assessment keys to the concerns. The concerns content is added to the concerns key in the inventory record of the virtual machine (VM). The web console displays the content of the concerns key for each VM.

The following .rego file example checks for distributed resource scheduling enabled (has_drs_enabled) in the cluster of a VMware VM: 

package io.konveyor.forklift.vmware

has_drs_enabled {
    input.host.cluster.drsEnabled
}

concerns[flag] {
    has_drs_enabled
    flag := {
        "category": "Information",
        "label": "VM running in a DRS-enabled cluster",
        "assessment": "Distributed resource scheduling is not currently supported by OpenShift Virtualization. The VM can be migrated but it will not have this feature in the target environment."
    }
}

where:

package io.konveyor.forklift.vmware
Is the package namespace in this example, The package namespaces are io.konveyor.forklift.vmware for VMware and io.konveyor.forklift.ovirt for Red Hat Virtualization.
input.host.cluster.drsEnabled
Is the query parameter in this example. Query parameters are based on the input key of the Validation service JSON.

9.4.2. Checking the default validation rules
Copy link

Before you create a custom rule, you must check the default rules of the Validation service to ensure that you do not create a rule that redefines an existing default value.

Example: If a default rule contains the line default valid_input = false and you create a custom rule that contains the line default valid_input = true, the Validation service will not start.

Procedure

  1. Connect to the terminal of the Validation pod: 

    $ oc rsh <validation_pod>

  2. Go to the OPA policies directory for your provider: 

    $ cd /usr/share/opa/policies/io/konveyor/forklift/<provider>

    where:

    <provider>
    Specifies the <provider>. Valid options: vmware or ovirt.

  3. Search for the default policies: 

    $ grep -R "default" *

9.4.3. Creating validation rules
Copy link

To ensure that your custom validation rules persist across pod restarts, scaling events, and Migration Toolkit for Virtualization (MTV) upgrades, deploy the rules by using a ConfigMap Custom Resource (CR). By default, the Validation service reads validation rules from a ConfigMap named forklift-validation-config in the openshift-mtv namespace. You can optionally customize the ConfigMap name by updating the forklift-controller CR.

Important
  • If you create a rule with the same name as an existing rule, the Validation service performs an OR operation with the rules.
  • If you create a rule that contradicts a default rule, the Validation service will not start.

Validation rules are based on VM attributes collected by the Provider Inventory service. The Provider Inventory service presents provider-specific VM properties as simplified attributes for the validation engine. You can then create Rego queries based on the attributes, and add the queries to the ConfigMap CR to apply validation rules across different source environments.

For example, in a validation rule that checks if a VMware VM has NUMA node affinity configured, you have these elements:

  • VMware API path: MOR:VirtualMachine.config.extraConfig["numa.nodeAffinity"].

  • Provider Inventory service attribute with a list value:

    Expand
    Inventory AttributeExample Value

    numa.nodeAffinity

    ["True"] or [] (empty list if not configured)

  • Rego query based on the attribute: 

    count(input.numaNodeAffinity) != 0

    For information about Rego files, see About Rego files.

Procedure

  1. Create a ConfigMap named forklift-validation-config in the openshift-mtv namespace:

    Note

    If you want to use a different ConfigMap name, you must also configure the forklift-controller CR. For more information, see step 2.

    Example: 

    $ cat << EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: forklift-validation-config
  namespace: openshift-mtv
data:
  vmware_multiple_disks.rego: |-
    package <provider_package>

    has_multiple_disks {
      count(input.disks) > 1
    }

    concerns[flag] {
      has_multiple_disks
        flag := {
          "category": "<Information>",
          "label": "Multiple disks detected",
          "assessment": "Multiple disks detected on this VM."
        }
    }
EOF
    • <provider_package>: The provider package name. Valid values are io.konveyor.forklift.vmware for VMware and io.konveyor.forklift.ovirt for Red Hat Virtualization.
    • count(input.disks): Your Rego query.
    • category: Valid values are Critical, Warning, and Information.

  2. Optional: If you are using a custom ConfigMap name instead of the default forklift-validation-config, add the validation_configmap_name parameter to the spec section of the ForkliftController CR: 

    spec:
  ...
  validation_configmap_name: <custom_configmap_name>
  ...
    • Replace <custom_configmap_name> with the name of your ConfigMap.

  3. Stop the Validation pod by scaling the forklift-validation deployment to 0: 

    $ oc scale -n openshift-mtv --replicas=0 deployment/forklift-validation

  4. Start the Validation pod by scaling the forklift-validation deployment to 1: 

    $ oc scale -n openshift-mtv --replicas=1 deployment/forklift-validation

  5. Check the Validation pod log to verify that the pod started: 

    $ oc logs -f <validation_pod>

    If the custom rule conflicts with a default rule, the Validation pod does not start.

  6. Remove the source provider: 

    $ oc delete provider <provider> -n openshift-mtv

  7. Add the source provider to apply the new rule. For information about adding a source provider, see the sections about adding a source provider in Chapters 10-14 of Planning your migration to Red Hat OpenShift Virtualization: 

    $ cat << EOF | oc apply -f -
apiVersion: forklift.konveyor.io/v1beta1
kind: Provider
metadata:
  name: <provider>
  namespace: openshift-mtv
spec:
  type: <provider_type>
  url: <api_end_point>
  secret:
    name: <secret>
    namespace: openshift-mtv
EOF
    • <provider_type>: Valid values are ovirt, vsphere, and openstack.
    • <api_end_point>: The API endpoint URL, for example, https://<vCenter_host>/sdk for vSphere, https://<engine_host>/ovirt-engine/api for RHV, or https://<identity_service>/v3 for OpenStack.
    • <secret>: The name of the provider Secret CR.

Next step

Update the inventory rules version after creating a custom rule so that the Provider Inventory service detects the changes and validates the VMs. For more information, see Updating the inventory rules version.

9.4.4. Updating the inventory rules version
Copy link

You must update the inventory rules version each time you update the rules so that the Provider Inventory service detects the changes and triggers the Validation service.

The rules version is recorded in a rules_version.rego file for each provider.

Procedure

  1. Retrieve the current rules version: 

    $ GET https://forklift-validation/v1/data/io/konveyor/forklift/<provider>/rules_version

    The output looks like the following example: 

    {
   "result": {
       "rules_version": 5
   }
}

  2. Connect to the terminal of the Validation pod: 

    $ oc rsh <validation_pod>
  3. Update the rules version in the /usr/share/opa/policies/io/konveyor/forklift/<provider>/rules_version.rego file.
  4. Log out of the Validation pod terminal.

  5. Verify the updated rules version: 

    $ GET https://forklift-validation/v1/data/io/konveyor/forklift/<provider>/rules_version

    The output looks like the following example: 

    {
   "result": {
       "rules_version": 6
   }
}

9.4.5. Retrieving the Inventory service JSON
Copy link

You retrieve the Inventory service JSON by sending an Inventory service query to a virtual machine (VM). The output contains an "input" key, which contains the inventory attributes that are queried by the Validation service rules.

You can create a validation rule based on any attribute in the "input" key, for example, input.snapshot.kind.

Procedure

  1. Retrieve the routes for the project: 

    oc get route -n openshift-mtv

  2. Retrieve the Inventory service route: 

    $ oc get route <inventory_service> -n openshift-mtv

  3. Retrieve the access token: 

    $ TOKEN=$(oc whoami -t)

  4. Trigger an HTTP GET request (for example, using Curl): 

    $ curl -H "Authorization: Bearer $TOKEN" https://<inventory_service_route>/providers -k

  5. Retrieve the UUID of a provider: 

    $ curl -H "Authorization: Bearer $TOKEN"  https://<inventory_service_route>/providers/<provider> -k

    where:

    <provider>
    Specifies the type of provider. Allowed values are vsphere, ovirt, and openstack.

  6. Retrieve the VMs of a provider: 

    $ curl -H "Authorization: Bearer $TOKEN"  https://<inventory_service_route>/providers/<provider>/<UUID>/vms -k

  7. Retrieve the details of a VM: 

    $ curl -H "Authorization: Bearer $TOKEN"  https://<inventory_service_route>/providers/<provider>/<UUID>/workloads/<vm> -k

    Example output

    {
    "input": {
        "selfLink": "providers/vsphere/c872d364-d62b-46f0-bd42-16799f40324e/workloads/vm-431",
        "id": "vm-431",
        "parent": {
            "kind": "Folder",
            "id": "group-v22"
        },
        "revision": 1,
        "name": "iscsi-target",
        "revisionValidated": 1,
        "isTemplate": false,
        "networks": [
            {
                "kind": "Network",
                "id": "network-31"
            },
            {
                "kind": "Network",
                "id": "network-33"
            }
        ],
        "disks": [
            {
                "key": 2000,
                "file": "[iSCSI_Datastore] iscsi-target/iscsi-target-000001.vmdk",
                "datastore": {
                    "kind": "Datastore",
                    "id": "datastore-63"
                },
                "capacity": 17179869184,
                "shared": false,
                "rdm": false
            },
            {
                "key": 2001,
                "file": "[iSCSI_Datastore] iscsi-target/iscsi-target_1-000001.vmdk",
                "datastore": {
                    "kind": "Datastore",
                    "id": "datastore-63"
                },
                "capacity": 10737418240,
                "shared": false,
                "rdm": false
            }
        ],
        "concerns": [],
        "policyVersion": 5,
        "uuid": "42256329-8c3a-2a82-54fd-01d845a8bf49",
        "firmware": "bios",
        "powerState": "poweredOn",
        "connectionState": "connected",
        "snapshot": {
            "kind": "VirtualMachineSnapshot",
            "id": "snapshot-3034"
        },
        "changeTrackingEnabled": false,
        "cpuAffinity": [
            0,
            2
        ],
        "cpuHotAddEnabled": true,
        "cpuHotRemoveEnabled": false,
        "memoryHotAddEnabled": false,
        "faultToleranceEnabled": false,
        "cpuCount": 2,
        "coresPerSocket": 1,
        "memoryMB": 2048,
        "guestName": "Red Hat Enterprise Linux 7 (64-bit)",
        "balloonedMemory": 0,
        "ipAddress": "10.19.2.96",
        "storageUsed": 30436770129,
        "numaNodeAffinity": [
            "0",
            "1"
        ],
        "devices": [
            {
                "kind": "RealUSBController"
            }
        ],
        "host": {
            "id": "host-29",
            "parent": {
                "kind": "Cluster",
                "id": "domain-c26"
            },
            "revision": 1,
            "name": "IP address or host name of the vCenter host or RHV Engine host",
            "selfLink": "providers/vsphere/c872d364-d62b-46f0-bd42-16799f40324e/hosts/host-29",
            "status": "green",
            "inMaintenance": false,
            "managementServerIp": "10.19.2.96",
            "thumbprint": <thumbprint>,
            "timezone": "UTC",
            "cpuSockets": 2,
            "cpuCores": 16,
            "productName": "VMware ESXi",
            "productVersion": "6.5.0",
            "networking": {
                "pNICs": [
                    {
                        "key": "key-vim.host.PhysicalNic-vmnic0",
                        "linkSpeed": 10000
                    },
                    {
                        "key": "key-vim.host.PhysicalNic-vmnic1",
                        "linkSpeed": 10000
                    },
                    {
                        "key": "key-vim.host.PhysicalNic-vmnic2",
                        "linkSpeed": 10000
                    },
                    {
                        "key": "key-vim.host.PhysicalNic-vmnic3",
                        "linkSpeed": 10000
                    }
                ],
                "vNICs": [
                    {
                        "key": "key-vim.host.VirtualNic-vmk2",
                        "portGroup": "VM_Migration",
                        "dPortGroup": "",
                        "ipAddress": "192.168.79.13",
                        "subnetMask": "255.255.255.0",
                        "mtu": 9000
                    },
                    {
                        "key": "key-vim.host.VirtualNic-vmk0",
                        "portGroup": "Management Network",
                        "dPortGroup": "",
                        "ipAddress": "10.19.2.13",
                        "subnetMask": "255.255.255.128",
                        "mtu": 1500
                    },
                    {
                        "key": "key-vim.host.VirtualNic-vmk1",
                        "portGroup": "Storage Network",
                        "dPortGroup": "",
                        "ipAddress": "172.31.2.13",
                        "subnetMask": "255.255.0.0",
                        "mtu": 1500
                    },
                    {
                        "key": "key-vim.host.VirtualNic-vmk3",
                        "portGroup": "",
                        "dPortGroup": "dvportgroup-48",
                        "ipAddress": "192.168.61.13",
                        "subnetMask": "255.255.255.0",
                        "mtu": 1500
                    },
                    {
                        "key": "key-vim.host.VirtualNic-vmk4",
                        "portGroup": "VM_DHCP_Network",
                        "dPortGroup": "",
                        "ipAddress": "10.19.2.231",
                        "subnetMask": "255.255.255.128",
                        "mtu": 1500
                    }
                ],
                "portGroups": [
                    {
                        "key": "key-vim.host.PortGroup-VM Network",
                        "name": "VM Network",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch0"
                    },
                    {
                        "key": "key-vim.host.PortGroup-Management Network",
                        "name": "Management Network",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch0"
                    },
                    {
                        "key": "key-vim.host.PortGroup-VM_10G_Network",
                        "name": "VM_10G_Network",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch1"
                    },
                    {
                        "key": "key-vim.host.PortGroup-VM_Storage",
                        "name": "VM_Storage",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch1"
                    },
                    {
                        "key": "key-vim.host.PortGroup-VM_DHCP_Network",
                        "name": "VM_DHCP_Network",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch1"
                    },
                    {
                        "key": "key-vim.host.PortGroup-Storage Network",
                        "name": "Storage Network",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch1"
                    },
                    {
                        "key": "key-vim.host.PortGroup-VM_Isolated_67",
                        "name": "VM_Isolated_67",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch2"
                    },
                    {
                        "key": "key-vim.host.PortGroup-VM_Migration",
                        "name": "VM_Migration",
                        "vSwitch": "key-vim.host.VirtualSwitch-vSwitch2"
                    }
                ],
                "switches": [
                    {
                        "key": "key-vim.host.VirtualSwitch-vSwitch0",
                        "name": "vSwitch0",
                        "portGroups": [
                            "key-vim.host.PortGroup-VM Network",
                            "key-vim.host.PortGroup-Management Network"
                        ],
                        "pNICs": [
                            "key-vim.host.PhysicalNic-vmnic4"
                        ]
                    },
                    {
                        "key": "key-vim.host.VirtualSwitch-vSwitch1",
                        "name": "vSwitch1",
                        "portGroups": [
                            "key-vim.host.PortGroup-VM_10G_Network",
                            "key-vim.host.PortGroup-VM_Storage",
                            "key-vim.host.PortGroup-VM_DHCP_Network",
                            "key-vim.host.PortGroup-Storage Network"
                        ],
                        "pNICs": [
                            "key-vim.host.PhysicalNic-vmnic2",
                            "key-vim.host.PhysicalNic-vmnic0"
                        ]
                    },
                    {
                        "key": "key-vim.host.VirtualSwitch-vSwitch2",
                        "name": "vSwitch2",
                        "portGroups": [
                            "key-vim.host.PortGroup-VM_Isolated_67",
                            "key-vim.host.PortGroup-VM_Migration"
                        ],
                        "pNICs": [
                            "key-vim.host.PhysicalNic-vmnic3",
                            "key-vim.host.PhysicalNic-vmnic1"
                        ]
                    }
                ]
            },
            "networks": [
                {
                    "kind": "Network",
                    "id": "network-31"
                },
                {
                    "kind": "Network",
                    "id": "network-34"
                },
                {
                    "kind": "Network",
                    "id": "network-57"
                },
                {
                    "kind": "Network",
                    "id": "network-33"
                },
                {
                    "kind": "Network",
                    "id": "dvportgroup-47"
                }
            ],
            "datastores": [
                {
                    "kind": "Datastore",
                    "id": "datastore-35"
                },
                {
                    "kind": "Datastore",
                    "id": "datastore-63"
                }
            ],
            "vms": null,
            "networkAdapters": [],
            "cluster": {
                "id": "domain-c26",
                "parent": {
                    "kind": "Folder",
                    "id": "group-h23"
                },
                "revision": 1,
                "name": "mycluster",
                "selfLink": "providers/vsphere/c872d364-d62b-46f0-bd42-16799f40324e/clusters/domain-c26",
                "folder": "group-h23",
                "networks": [
                    {
                        "kind": "Network",
                        "id": "network-31"
                    },
                    {
                        "kind": "Network",
                        "id": "network-34"
                    },
                    {
                        "kind": "Network",
                        "id": "network-57"
                    },
                    {
                        "kind": "Network",
                        "id": "network-33"
                    },
                    {
                        "kind": "Network",
                        "id": "dvportgroup-47"
                    }
                ],
                "datastores": [
                    {
                        "kind": "Datastore",
                        "id": "datastore-35"
                    },
                    {
                        "kind": "Datastore",
                        "id": "datastore-63"
                    }
                ],
                "hosts": [
                    {
                        "kind": "Host",
                        "id": "host-44"
                    },
                    {
                        "kind": "Host",
                        "id": "host-29"
                    }
                ],
                "dasEnabled": false,
                "dasVms": [],
                "drsEnabled": true,
                "drsBehavior": "fullyAutomated",
                "drsVms": [],
                "datacenter": null
            }
        }
    }
}

9.5. About hooks for MTV migration plans
Copy link

You can add hooks to an Migration Toolkit for Virtualization (MTV) migration plan to perform automated operations on a VM, either before or after you migrate it.

You can add hooks to Migration Toolkit for Virtualization (MTV) migration plans by using either the MTV CLI or the MTV user interface, which is located in the Red Hat OpenShift web console.

Hook types
  • Pre-migration hooks: Hooks that perform operations on a VM that is located on a provider. This prepares the VM for migration.
  • Post-migration hooks: Hooks that perform operations on a VM that has migrated to OpenShift Virtualization.
Hook configuration
  • Default hook image: The default hook image for an MTV hook is {hook-runner-image}. The image is based on the Ansible Runner image with the addition of python-openshift to provide Ansible Kubernetes resources and a recent oc binary.
  • Hook execution: An Ansible Playbook that is provided as part of a migration hook is mounted into the hook container as a ConfigMap. The hook container is run as a job on the relevant cluster in the openshift-mtv namespace. When you add a hook, you must specify the name of the hook and whether it is a pre-migration hook or a post-migration hook.
  • Service account: You can optionally specify a service account when adding a hook. When you add a hook by using the Red Hat OpenShift web console, you can specify the service account name in the Service account field when creating or editing the migration plan. When you add a hook by using the CLI, you can specify the service account in the Hook CR. If you specify a service account, it must have the appropriate RBAC permissions to manage cluster resources and at least write access for the openshift-mtv namespace where hooks execute.
Important

For a hook to run on a VM, the VM must be started and available using SSH.

9.5.1. Migration hook workflow
Copy link

The illustration that follows shows the general process of using a migration hook.

Figure 9.1. Adding a hook to a migration plan

Adding a hook to a migration plan

Process:

  1. Input your Ansible hook and credentials.

    1. Input an Ansible hook image to the MTV controller using either the UI or the CLI.

      • In the UI, specify the ansible-runner and enter the playbook.yml that contains the hook.
      • In the CLI, input the hook image, which specifies the playbook that runs the hook.

    2. If you need additional data to run the playbook inside the pod, such as SSH data, create a Secret that contains credentials for the VM. The Secret is not mounted to the pod, but is called by the playbook.

      Note

      This Secret is not the same as the Secret CR that contains the credentials of your source provider.

  2. The MTV controller creates the ConfigMap, which contains:

    • workload.yml, which contains information about the VMs.
    • playbook.yml, the raw string playbook you want to run.

    • plan.yml, which is the Plan CR.

      The ConfigMap contains the name of the VM and instructs the playbook what to do.

  3. The MTV controller creates a job that starts the user-specified image.

    1. Mounts the ConfigMap to the container.

      The Ansible hook imports the Secret that the user previously entered.

  4. The job runs a pre-migration hook or a post-migration hook as follows:

    1. For a pre-migration hook, the job logs into the VMs on the source provider using SSH and runs the hook.
    2. For a post-migration hook, the job logs into the VMs on OpenShift Virtualization using SSH and runs the hook.

You can add a migration hook to an existing migration plan by using the Red Hat OpenShift web console. For example, you can create a hook to install the cloud-init service on a VM and write a file before migration. You can optionally specify a service account for the hook directly in the web console.

You can run one pre-migration hook, one post-migration hook, or one of each per migration plan.

Prerequisites

  • Migration plan.
  • Migration hook file, whose contents you copy and paste into the web console.
  • File containing the Secret for the source provider.
  • (Optional) Red Hat OpenShift service account called by the hook. The service account must have at least write access for the openshift-mtv namespace where hooks execute. For information about creating a service account, see Understanding and creating service accounts.
  • SSH access for VMs you want to migrate with the public key installed on the VMs.
  • VMs running on Microsoft Server only: Remote Execution enabled.

Procedure

  1. In the Red Hat OpenShift web console, click Migration for Virtualization > Migration plans and then click the migration plan you want to add the hook to.
  2. Click Hooks.

  3. For a pre-migration hook, perform the following steps:

    1. In the Pre migration hook section, toggle the Enable hook switch to Enable pre migration hook.
    2. Enter the Hook runner image. If you are specifying the spec.playbook, you need to use an image that has an ansible-runner.
    3. Optional: Enter the Service account name. The service account must have the necessary RBAC permissions to manage cluster resources and at least write access for the openshift-mtv namespace where hooks execute.
    4. Paste your hook as a YAML file in the Ansible playbook text box.

  4. For a post-migration hook, perform the following steps:

    1. In the Post migration hook, toggle the Enable hook switch to Enable post migration hook.
    2. Enter the Hook runner image. If you are specifying the spec.playbook, you need to use an image that has an ansible-runner.
    3. Optional: Enter the Service account name. The service account must have the necessary RBAC permissions to manage cluster resources and at least write access for the openshift-mtv namespace where hooks execute.
    4. Paste your hook as a YAML file in the Ansible playbook text box.

  5. At the top of the tab, click Update hooks.

    The following example hook ensures that the VM can be accessed using SSH, creates an SSH key, and runs two tasks: stopping the MariaDB database and generating a text file. 

    - name: Main
  hosts: localhost
  vars_files:
    - plan.yml
    - workload.yml
  tasks:
  - k8s_info:
      api_version: v1
      kind: Secret
      name: privkey
      namespace: openshift-mtv
    register: ssh_credentials

  - name: Ensure SSH directory exists
    file:
      path: ~/.ssh
      state: directory
      mode: 0750

  - name: Create SSH key
    copy:
      dest: ~/.ssh/id_rsa
      content: "{{ ssh_credentials.resources[0].data.key | b64decode }}"
      mode: 0600

  - add_host:
      name: "{{ vm.ipaddress }}"  # ALT "{{ vm.guestnetworks[2].ip }}"
      ansible_user: root
      groups: vms

- hosts: vms
  vars_files:
    - plan.yml
    - workload.yml
  tasks:
  - name: Stop MariaDB
    service:
      name: mariadb
      state: stopped

  - name: Create Test File
    copy:
      dest: /premigration.txt
      content: "Migration from {{ provider.source.name }}
                of {{ vm.vm1.vm0.id }} has finished\n"
      mode: 0644

You can use a Hook CR to add a pre-migration hook or a post-migration hook to an existing migration plan by using the Migration Toolkit for Virtualization (MTV) CLI. For example, you can create a Hook custom resource (CR) to install the cloud-init service on a VM and write a file before migration.

You can run one pre-migration hook, one post-migration hook, or one of each per migration plan. Each hook needs its own Hook CR, but a Plan CR contains data for all the hooks it uses. You can retrieve additional information stored in a secret or in a ConfigMap by using a k8s module.

Prerequisites

  • Migration plan.
  • Migration hook image or the playbook containing the hook image.
  • File containing the Secret for the source provider.
  • (Optional) Red Hat OpenShift service account called by the hook. The service account must have at least write access for the openshift-mtv namespace where hooks execute. For information about creating a service account, see Understanding and creating service accounts.
  • SSH access for VMs you want to migrate with the public key installed on the VMs.
  • VMs running on Microsoft Server only: Remote Execution enabled.

Procedure

  1. If needed, create a Secret with an SSH private key for the VM.

    1. Choose an existing key or generate a key pair.
    2. Install the public key on the VM.

    3. Encode the private key in the Secret to base64. 

      apiVersion: v1
data:
  key: VGhpcyB3YXMgZ2Vu...
kind: Secret
metadata:
  name: ssh-credentials
  namespace: openshift-mtv
type: Opaque

  2. Encode your playbook by concatenating a file and piping it for Base64 encoding, for example: 

    $ cat playbook.yml | base64 -w0

  3. Create a Hook CR: 

    $  cat << EOF | oc apply -f -
apiVersion: forklift.konveyor.io/v1beta1
kind: Hook
metadata:
  name: <hook>
  namespace: <namespace>
spec:
  image: registry.redhat.io/rhmtc/openshift-migration-hook-runner-rhel8
  serviceAccount: <service_account>
  playbook: |
    <playbook>
EOF

    where:

    <service_account>

    Optional: Specifies the Red Hat OpenShift service account. The serviceAccount must be provided if you want to manipulate any resources of the cluster.

    Note

    You can set the service account directly in the Red Hat OpenShift web console when creating or editing a migration plan by using the Service account field. See Adding a migration hook using the web console for the simplified UI workflow. The CLI/YAML method documented here is still supported for advanced use cases and automation.

    <playbook>

    Specifies the Base64-encoded Ansible Playbook to use. For example, LS0tCi0gbm.... If you specify a playbook, the image must include an ansible-runner. You can use the default hook-runner image or specify a custom image. If you specify a custom image, you do not have to specify a playbook.

    Note

    To decode an attached playbook, retrieve the resource with custom output and pipe it to base64. For example: 

    $ oc get -n openshift-mtv hook playbook -o \
    go-template='{{ .spec.playbook }}' | base64 -d

  4. In the Plan CR of the migration, for each VM, add the following section to the end of the CR: 

      vms:
    - id: <vm_id>
      hooks:
        - hook:
            namespace: <namespace>
            name: <name_of_hook>
          step: <type_of_hook>

    where:

    <type_of_hook>

    Specifies the type of hook. Options are PreHook, to run the hook before the migration, and PostHook, to run the hook after the migration. In order for a PreHook to run on a VM, the VM must be started and available using SSH.

    The following example hook ensures that the VM can be accessed using SSH, creates an SSH key, and runs two tasks: stopping the MariaDB database and generating a text file. 

    - name: Main
  hosts: localhost
  vars_files:
    - plan.yml
    - workload.yml
  tasks:
  - k8s_info:
      api_version: v1
      kind: Secret
      name: privkey
      namespace: openshift-mtv
    register: ssh_credentials

  - name: Ensure SSH directory exists
    file:
      path: ~/.ssh
      state: directory
      mode: 0750

  - name: Create SSH key
    copy:
      dest: ~/.ssh/id_rsa
      content: "{{ ssh_credentials.resources[0].data.key | b64decode }}"
      mode: 0600

  - add_host:
      name: "{{ vm.ipaddress }}"  # ALT "{{ vm.guestnetworks[2].ip }}"
      ansible_user: root
      groups: vms

- hosts: vms
  vars_files:
    - plan.yml
    - workload.yml
  tasks:
  - name: Stop MariaDB
    service:
      name: mariadb
      state: stopped

  - name: Create Test File
    copy:
      dest: /premigration.txt
      content: "Migration from {{ provider.source.name }}
                of {{ vm.vm1.vm0.id }} has finished\n"
      mode: 0644

9.6. About user defined networks
Copy link

Beginning with Migration Toolkit for Virtualization 2.10, you can use a user-defined network (UDN) as your default network for migrations from all source providers, except when the source provider is also OpenShift Virtualization. UDNs are not supported for OpenShift-to-OpenShift migrations because the network configuration is preserved from the source cluster. This flexibility allows you to migrate virtual machines (VMs) to OpenShift Virtualization more consistently.

MTV has been redesigned to make it easy for you to migrate VMs to UDN namespaces. Once you configure your UDN in OpenShift Virtualization, you can specify it as your default network in the migration plan mapping. MTV is now able to distinguish a UDN from a conventional pod network.

You can use this feature for creating migration plans by using the Red Hat OpenShift web console or by using the MTV command-line interface. The procedures for creating migration plans have been updated to include the new feature.

9.7. Scheduling target VMs
Copy link

By default, OpenShift Virtualization assigns the destination nodes of virtual machines (VMs) during migration. However, you can use the scheduling target VMs feature to define the destination nodes and apply specific conditions to schedule when the VMs are switched from pending to on.

9.7.1. About scheduling target VMs
Copy link

Starting with Migration Toolkit for Virtualization (MTV) 2.10, you can use the target VM scheduling feature to direct MTV to migrate virtual machines (VMs) to specific nodes of OpenShift Virtualization as well as to schedule when to power on the VMs. Using the feature, you can design and enforce rules that you set using either the UI or command-line interface.

Previously, when you migrated VMs to OpenShift Virtualization, OpenShift Virtualization automatically determined the node the VMs would be migrated to. Although this served many customers' needs, there are certain situations in which it is useful to be able to specify the target node of a VM or the conditions under which the VM is powered on, regardless of the type of migration involved.

9.7.2. Target VM scheduling prerequisites
Copy link

Migrations that use target VM scheduling require the following prerequisites, in addition to the prerequisites for your source provider:

  • Migration Toolkit for Virtualization (MTV) 2.10 or later.
  • Version of OpenShift Virtualization that is compatible with your version of MTV. For MTV 2.10, the compatible versions of OpenShift Virtualization are 4.18, 4.19, and 4.20 only.
  • cluster-admin or equivalent security privileges that allow managing VirtualMachineInstance objects and associated Kubernetes scheduling primitives.

9.7.3. Target VM scheduling options
Copy link

You can use the following options to schedule when your target VMs are powered on:

  • Node Selector rule: This is the simplest rule. You define a set of mandatory exact match key-value label pairs that the target node must possess. If no node on the cluster has all the labels specified, the VM is not scheduled and it remains in a Pending state until there is space on a node that fits the key-value label pairs.

  • Affinity and Anti-Affinity rules: Node Affinity rules let you schedule VMs to run on selected nodes or workloads (pods). Node Anti-affinity rules let you prevent VMs from being scheduled to run on selected workloads (pods).

    Node Affinity and Node Anti-Affinity rules offer more flexible placement control than rigid Node Selector rules, because they support conditionals such as In, NotIn.

    Additionally, Affinity rules and Anti-Affinity rules allow you to include both hard and soft conditions in the same rule. A hard condition is a requirement, and a soft condition is a preference.

Note

Affinity rules are supported by MTV at both the node and the workload (pod) levels, but Anti-Affinity rules are supported at the workload (pod) level only.

  • Custom Scheduler Name: If your OpenShift Virtualization environment uses a secondary or specialized scheduler, in addition to the default kube-scheduler, to handle specific workload types, you can instruct MTV to apply this custom scheduler name to the target VM’s manifest. This directs the VM to use the specialized logic designed for that workload. This feature is implemented by using the VM target label feature.

By integrating any of these three types of controls into your migration plan, you ensure that the complex scheduling logic required for modern applications is defined upfront, preventing post-migration performance degradation or unexpected scheduling errors.

Important

Any scheduling rule applied in a migration plan applies to all VMs in it.

You can use the command-line interface (CLI) to tell Migration Toolkit for Virtualization (MTV) to migrate virtual machines (VMs) to specific nodes or workloads (pods) of OpenShift Virtualization as well as to schedule when the VMs are powered on.

The MTV CLI supports the following scheduling-related labels, all of which are added to the Plan CR:

  • targetAffinity: Implements placement policies such as co-locating related workloads or, for disaster recovery, ensuring that specific VMs are migrated to different nodes. This type of label uses hard (requirements) and soft (preferences) conditions combined with logical operators, such as and, or, and not, to provide greater flexibility than the targetLabelSelector label discussed following.
  • targetLabels: Applies organizational or operational labels to migrated VMs for identification and management.
  • targetNodeSelector: Ensures VMs are scheduled on nodes that are an exact match for key-value pairs you create. This type of label is often used for nodes with special capabilities, such as GPU nodes or storage nodes.
Important

System-managed labels, such as migration, plan, VM ID, or application labels, override any user-defined labels.

Prerequisites

Migrations that use target VM scheduling require the following prerequisites, in addition to the prerequisites for your source provider:

  • Migration Toolkit for Virtualization (MTV) 2.10 or later.
  • Version of OpenShift Virtualization that is compatible with your version of MTV. For MTV 2.10, the compatible versions of OpenShift Virtualization are 4.18, 4.19, and 4.20 only.
  • cluster-admin or equivalent security privileges that allow managing VirtualMachineInstance objects and associated Kubernetes scheduling primitives.

Procedure

  1. Create custom resources (CR)s for the migration according to the procedure for the provider.

  2. In the Plan CR, add the following labels before spec:targetNamespace. All are optional. 

    ...
  targetAffinity: <affinity rule, which can be quite complex, is entered in lines following this label. See example that follows>
  targetLabels:
    label: <label>
  targetNodeSelector:
    <key>:<value>
  targetNamespace:<target_namespace>
...

    Example:

    The following scheduling rule migrates the VMs in the plan to different nodes for disaster recovery: 

  targetLabels:
    label: test1
  targetAffinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: label
                operator: In
                values:
                  - test1
          topologyKey: kubernetes.io/hostname

Result

As a result of the preceding rule, the VMs are migrated accordingly to the resulting spec: 

spec:
  runStrategy: Always
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: mtv-rhel8-sanity-ceph-rbd
        label: test1
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: label
                    operator: In
                    values:
                      - test1
              topologyKey: kubernetes.io/hostname

You can use the Migration Toolkit for Virtualization (MTV) user interface, which is located in the Red Hat OpenShift web console, to tell Migration Toolkit for Virtualization (MTV) to migrate virtual machines (VMs) to specific nodes or workloads (pods) of OpenShift Virtualization as well as to schedule when the VMs are powered on.

The Virtualization section of the Red Hat OpenShift web console supports the following options for scheduling target VMs:

  • VM target node selector: Ensures VMs are scheduled on nodes that are an exact match for key-value pairs you create. This type of label is often used for nodes with special capabilities, such as GPU nodes or storage nodes.
  • VM target labels: Applies organizational or operational labels to migrated VMs for identification and management.

  • VM target affinity rules: Implements placement policies such as co-locating related workloads or, for disaster recovery, ensuring that specific VMs are migrated to different nodes. This type of rule uses hard (requirements) and soft (preferences) conditions combined with logical operators, such as Exists or DoesNotExist instead of using the rigid key-value pairs used by a VM target node selector. As a result, target affinity rules are more flexible than target node selector rules.

    The MTV UI supports the following affinity rules:

    • Node affinity rules
    • Workload (pod) affinity and anti-affinity rules

You configure target VM scheduling options on the Plan details page of the relevant migration plan. The options apply to all VMs that are included in that migration.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top