Chapter 2. Migration Toolkit for Virtualization 2.5

Migration from OpenStack moves to being a fully supported feature

In this version of MTV, migration using OpenStack source providers graduated from a Technology Preview feature to a fully supported feature.

Disabling FIPS

MTV enables migrations from vSphere source providers by not enforcing Enterprise Master Secret (EMS). This enables migrating from all vSphere versions that MTV supports, including migrations that do not meet 2023 FIPS requirements.

Integration of the create and update provider user interface

The user interface of the create and update providers now aligns with the look and feel of the Red Hat OpenShift web console and displays up-to-date data.

Standalone UI

The old UI of MTV 2.3 cannot be enabled by setting feature_ui: true in ForkliftController anymore.

Support deployment on OpenShift 4.15

MTV 2.5.6 can be deployed on OpenShift 4.15 clusters.

Migration of OVA files from VMware vSphere

In MTV 2.6, you can migrate using Open Virtual Appliance (OVA) files that were created by VMware vSphere as source providers. (MTV-336)

Migrating VMs between Red Hat OpenShift clusters

In MTV 2.6, you can now use Red Hat OpenShift Virtualization provider as a source provider and a destination provider. You can migrate VMs from the cluster that MTV is deployed on to another cluster, or from a remote cluster to the cluster that MTV is deployed on. (MTV-571)

Migration of VMs with direct LUNs from RHV

During the migration from Red Hat Virtualization (RHV), direct Logical Units (LUNs) are detached from the source virtual machines and attached to the target virtual machines. Note that this mechanism does not work yet for Fibre Channel. (MTV-329)

Additional authentication methods for OpenStack

In addition to standard password authentication, MTV supports the following authentication methods: Token authentication and Application credential authentication. (MTV-539)

Validation rules for OpenStack

The validation service includes default validation rules for virtual machines from OpenStack. (MTV-508)

VDDK is now optional for VMware vSphere providers

You can now create the VMware vSphere source provider without specifying a VMware Virtual Disk Development Kit (VDDK) init image. It is strongly recommended you create a VDDK init image to accelerate migrations.

Deployment on OKE enabled

In MTV 2.5.3, deployment on OpenShift Kubernetes Engine (OKE) has been enabled. For more information, see About OpenShift Kubernetes Engine. (MTV-803)

Migration of VMs to destination storage classes with encrypted RBD now supported

In MTV 2.5.4, migration of VMs to destination storage classes that have encrypted RADOS Block Devices (RBD) volumes is now supported.

Deleting migration plan does not remove temporary resources

Deleting a migration plan does not remove temporary resources such as importer pods, conversion pods, config maps, secrets, failed VMs and data volumes. You must archive a migration plan before deleting it to clean up the temporary resources. (BZ#2018974)

Unclear error status message for VM with no operating system

The error status message for a VM with no operating system on the Plans page of the web console does not describe the reason for the failure. (BZ#22008846)

Migration of virtual machines with encrypted partitions fails during conversion

vSphere only: Migrations from RHV and OpenStack do not fail, but the encryption key may be missing on the target Red Hat OpenShift cluster.

Migration fails during precopy/cutover while performing a snapshot operation on the source VM

Warm migration from RHV fails if a snapshot operation is triggered and running on the source VM at the same time as the migration is scheduled. The migration does not wait for the snapshot operation to finish. (MTV-456)

Unable to schedule migrated VM with multiple disks to more than one storage classes of type hostPath

When migrating a VM with multiple disks to more than one storage classes of type hostPath, it might happen that a VM cannot be scheduled. Workaround: Use shared storage on the target Red Hat OpenShift cluster.

Non-supported guest operating systems in warm migrations

Warm migrations and migrations to remote Red Hat OpenShift clusters from vSphere do not support all types of guest operating systems that are supported in cold migrations to the local Red Hat OpenShift cluster. This is a consequence of using RHEL 8 in the former case and RHEL 9 in the latter case.
See Converting virtual machines from other hypervisors to KVM with virt-v2v in RHEL 7, RHEL 8, and RHEL 9 for the list of supported guest operating systems.

VMs from vSphere with RHEL 9 guest operating system can start with network interfaces that are down

When migrating VMs that are installed with RHEL 9 as guest operating system from vSphere, the network interfaces of the VMs could be disabled when they start in OpenShift Virtualization. (MTV-491)

Import OVA: ConnectionTestFailed message appears when adding OVA provider

When adding an OVA provider, the error message ConnectionTestFailed can appear, although the provider is created successfully. If the message does not disappear after a few minutes and the provider status does not move to Ready, this means that the ova server pod creation has failed. (MTV-671)

Left over ovirtvolumepopulator from failed migration causes plan to stay indefinitely in CopyDisks phase

An outdated ovirtvolumepopulator in the namespace, left over from an earlier failed migration, stops a new plan of the same VM when it transitions to CopyDisks phase. The plan remains in that phase indefinitely. (MTV-929)

Unclear error message when Forklift fails to build a PVC

The migration fails to build the Persistent Volume Claim (PVC) if the destination storage class does not have a configured storage profile. The forklift-controller raises an error message without a clear reason for failing to create a PVC. (MTV-928)

Flaw was found in jsrsasign package which is vulnerable to Observable Discrepancy

Versions of the package jsrsasign before 11.0.0, used in earlier releases of MTV, are vulnerable to Observable Discrepancy in the RSA PKCS1.5 or RSA-OAEP decryption process. This discrepancy means an attacker could decrypt ciphertexts by exploiting this vulnerability. However, exploiting this vulnerability requires the attacker to have access to a large number of ciphertexts encrypted with the same key. This issue has been resolved in MTV 2.5.5 by upgrading the package jsrasign to version 11.0.0.

Multiple HTTP/2 enabled web servers are vulnerable to a DDoS attack (Rapid Reset Attack)

A flaw was found in handling multiplexed streams in the HTTP/2 protocol. In previous releases of MTV, the HTTP/2 protocol allowed a denial of service (server resource consumption) because request cancellation could reset multiple streams quickly. The server had to set up and tear down the streams while not hitting any server-side limit for the maximum number of active streams per connection, which resulted in a denial of service due to server resource consumption.

Gin Web Framework does not properly sanitize filename parameter of Context.FileAttachment function

A flaw was found in the Gin-Gonic Gin Web Framework, used by MTV. The filename parameter of the Context.FileAttachment function was not properly sanitized. This flaw in the package could allow a remote attacker to bypass security restrictions caused by improper input validation by the filename parameter of the Context.FileAttachment function. A maliciously created filename could cause the Content-Disposition header to be sent with an unexpected filename value, or otherwise modify the Content-Disposition header.

CVE-2023-26144: mtv-console-plugin-container: graphql: Insufficient checks in the OverlappingFieldsCanBeMergedRule.ts

A flaw was found in the package GraphQL from 16.3.0 and before 16.8.1. This flaw means MTV versions before MTV 2.5.2 are vulnerable to Denial of Service (DoS) due to insufficient checks in the OverlappingFieldsCanBeMergedRule.ts file when parsing large queries. This issue may allow an attacker to degrade system performance. (MTV-712)

CVE-2023-45142: Memory leak found in the otelhttp handler of open-telemetry

A flaw was found in otelhttp handler of OpenTelemetry-Go. This flaw means MTV versions before MTV 2.5.3 are vulnerable to a memory leak caused by http.user_agent and http.method having unbound cardinality, which could allow a remote, unauthenticated attacker to exhaust the server’s memory by sending many malicious requests, affecting the availability. (MTV-795)

CVE-2023-39322: QUIC connections do not set an upper bound on the amount of data buffered when reading post-handshake messages

A flaw was found in Golang. This flaw means MTV versions before MTV 2.5.3 are vulnerable to QUIC connections not setting an upper bound on the amount of data buffered when reading post-handshake messages, allowing a malicious QUIC connection to cause unbounded memory growth. With the fix, connections now consistently reject messages larger than 65KiB in size. (MTV-708)

CVE-2023-39321: Processing an incomplete post-handshake message for a QUIC connection can cause a panic

A flaw was found in Golang. This flaw means MTV versions before MTV 2.5.3 are vulnerable to processing an incomplete post-handshake message for a QUIC connection, which causes a panic. (MTV-693)

CVE-2023-39319: Flaw in html/template package

A flaw was found in the Golang html/template package used in MTV. This flaw means MTV versions before MTV 2.5.3 are vulnerable, as the html/template package did not properly handle occurrences of <script, <!--, and </script within JavaScript literals in <script> contexts. This flaw could cause the template parser to improperly consider script contexts to be terminated early, causing actions to be improperly escaped, which could be leveraged to perform an XSS attack. (MTV-693)

CVE-2023-39318: Flaw in html/template package

A flaw was found in the Golang html/template package used in MTV. This flaw means MTV versions before MTV 2.5.3 are vulnerable as the html/template package did not properly handle HMTL-like "" comment tokens, nor hashbang \#! comment tokens. This flaw could cause the template parser to improperly interpret the contents of <script> contexts, causing actions to be improperly escaped, which could be leveraged to perform an XSS attack. (MTV-693)

Logs archive file downloaded from UI includes logs related to deleted migration plan/VM

In earlier releases of MTV 2.6, the log files downloaded from UI could contain logs that are related to an earlier migration plan. (MTV-783)

Extending a VM disk in RHV is not reflected in the MTV inventory

In earlier releases of MTV 2.6, the size of disks that are extended in RHV was not adequately monitored. This resulted in the inability to migrate virtual machines with extended disks from a RHV provider. (MTV-830)

Filesystem overhead configurable

In earlier releases of MTV 2.6, the filesystem overhead for new persistent volumes was hard-coded to 10%. The overhead was insufficient for certain filesystem types, resulting in failures during cold-migrations from RHV and OSP to the cluster where MTV is deployed. In other filesystem types, the hard-coded overhead was too high, resulting in excessive storage consumption.

Ensure up-to-date data is displayed in the create and update provider forms

In earlier releases of MTV, the create and update provider forms could have presented stale data.

Snapshots that are created during a migration in OpenStack are not deleted

In earlier releases of MTV, the Migration Controller service did not delete snapshots that were created during a migration of source virtual machines in OpenStack automatically.

RHV snapshots are not deleted after a successful migration

In earlier releases of MTV, the Migration Controller service did not delete snapshots automatically after a successful warm migration of a VM from RHV.

Warm migration fails when cutover conflicts with precopy

In earlier releases of MTV, the cutover operation failed when it was triggered while precopy was being performed. The VM was locked in RHV and therefore the ovirt-engine rejected the snapshot creation, or disk transfer, operation.

Warm migration fails when VM is locked

In earlier releases of MTV, triggering a warm migration while there was an ongoing operation in RHV that locked the VM caused the migration to fail because it could not trigger the snapshot creation.

Deleting migrated VM does not remove PVC and PV

In earlier releases of MTV, when removing a VM that was migrated, its persistent volume claims (PVCs) and physical volumes (PV) were not deleted.

PVC deletion hangs after archiving and deleting migration plan

In earlier releases of MTV, when a migration failed, its PVCs and PVs were not deleted as expected when its migration plan was archived and deleted.

VM with multiple disks can boot from a non-bootable disk after migration

In earlier releases of MTV, VM with multiple disks that were migrated might not have been able to boot on the target Red Hat OpenShift cluster.

Transfer network not taken into account for cold migrations from vSphere

In MTV releases 2.4.0-2.5.3, cold migrations from vSphere to the local cluster on which MTV was deployed did not take a specified transfer network into account. This issue is resolved in MTV 2.5.4. (MTV-846)

Fix migration of VMs with multi-boot guest operating system from vSphere

In MTV 2.5.6, the virt-v2v arguments include –root first, which mitigates an issue with multi-boot VMs where the pod fails. This is a fix for a regression that was introduced in MTV 2.4, in which the --root argument was dropped. (MTV-987)

Errors logged in populator pods are improved

In earlier releases of MTV 2.6, populator pods were always restarted on failure. This made it difficult to gather the logs from the failed pods. In MTV 2.5.3, the number of restarts of populator pods is limited to three times. On the third and final time, the populator pod remains in the fail status and its logs can then be easily gathered by must-gather and by forklift-controller to know this step has failed. (MTV-818)

npm IP package vulnerability

A vulnerability found in the Node.js Package Manager (npm) IP Package can allow an attacker to obtain sensitive information and obtain access to normally inaccessible resources. MTV-941

Flaw was found in the Golang net/http/internal package

A flaw was found in the versions of the Golang net/http/internal package, that were used in earlier releases of MTV. This flaw could allow a malicious user to send an HTTP request and cause the receiver to read more bytes from the network than are in the body (up to 1GiB), causing the receiver to fail reading the response, possibly leading to a Denial of Service (DoS). This issue has been resolved in MTV 2.5.6.

Upgrade from 2.4.0 fails

When upgrading from MTV 2.4.0 to a later version, the operation fails with an error that says the field spec.selector of deployment forklift-controller is immutable. Workaround: Remove the custom resource forklift-controller of type ForkliftController from the installed namespace, and recreate it. Refresh the Red Hat OpenShift console once the forklift-console-plugin pod runs to load the upgraded MTV web console. (MTV-518)