Migrating VMs to a Red Hat OpenStack Platform deployment
Migrating virtual machines to a Red Hat OpenStack Platform deployment
Abstract
Providing feedback on Red Hat documentation Copy linkLink copied to clipboard!
We appreciate your feedback. Tell us how we can improve the documentation.
To provide documentation feedback for Red Hat OpenStack Platform (RHOSP), create a Jira issue in the OSPRH Jira project.
Procedure
- Log in to the Red Hat Atlassian Jira.
- Click the following link to open a Create Issue page: Create issue
- Complete the Summary and Description fields. In the Description field, include the documentation URL, chapter or section number, and a detailed description of the issue.
- Click Create.
- Review the details of the bug you created.
Chapter 1. Migrating virtual machines overview Copy linkLink copied to clipboard!
You can use the Red Hat OpenStack VMware Migration toolkit to help you move cloud workloads from a VMWare environment to a Red Hat OpenStack Platform (RHOSP) environment.
1.1. Prerequisites Copy linkLink copied to clipboard!
- Source and destination cloud deployments.
- An instance of Red Hat Ansible Automation Environment 2.5 or later.
- Red Hat Enterprise Linux (RHEL) 9.6 or later QCOW image onboarded on the destination cloud for the conversion host.
- A host to run Ansible Builder.
- Network access for outbound HTTPS and inbound SSH.
RHOSP permissions that allow you to perform the following operations:
- Instance creation and deletion.
- Network resource management.
- Security group management.
- Key pair management.
- Floating IP allocation.
1.2. Validating and checking migration prerequisites Copy linkLink copied to clipboard!
Before you migrate VMware workloads to Red Hat OpenStack Platform (RHOSP), perform the following checks and validations:
- Ensure that all virtual machine (VM) disks are consolidated.
- Check the snapshot hierarchy depth and remove or consolidate unnecessary snapshots.
- If you use large disks, enable Change Block Tracking (CBT) on the VMs. CBT is a VMware vSphere feature that tracks which disk blocks of a virtual machine (VM) have changed since the last backup or snapshot. You can enable CBT in VMware vSphere.
- If you use CBT, ensure that VMware Tools is installed.
Ensure that you apply the following network configurations:
Expand Table 1.1. Network requirements Port / Protocol Direction Source / Destination 443 / TCP
Egress
VMware vCenter
902 / TCP
Egress
VMware ESXi hosts
22 / TCP
Ingress
Ansible Controller / Admin
10809 / TCP
Internal on host
Conversion host
Configure the appropriate VMware user Access Control Lists (ACL)s:
Expand Table 1.2. VMware user Access Control Lists (ACL)s Category Privilege Group Privileges Datastore
Browse Datastore
Virtual Machine
Guest operations
All
Provisioning
Allow disk access
Allow file access
Allow read-only disk access
Allow virtual machine downloadService configuration
Allow notifications
Allow polling of global event notifications
Read service configurationSnapshot management
Create snapshot
Remove snapshot
Rename snapshot
Revert to snapshot
1.3. Enabling CBT Copy linkLink copied to clipboard!
If you use large disks, enable Change Block Tracking (CBT) on the VMs to minimize downtime. CBT is a VMware vSphere feature that tracks which disk blocks of a virtual machine (VM) have changed since the last backup or snapshot.
Prerequisites
- VMware Tools must be installed on the VM.
- You must power off the VM or take a snapshot to apply the changes.
- vSphere 4.0 or later.
Procedure
Check VMware Tools status:
-
On the vSphere Client, select the VM and click the
Summarytab. -
Verify if the status of VMware Tools is
RunningorOK. -
(Optional) If the status of VMware Tools is
Not installedorNot running, right click the VM, selectGuest OS,Install VMware Tools, and follow the VMware Tools installation process for the guest OS.
-
On the vSphere Client, select the VM and click the
- Power off the VM.
Edit the VM settings and set the
stkEnabledparameter toTRUE:ctkEnabled = TRUESet
ctkEnabled = TRUEfor each disk on the VM:scsi0:0.ctkEnabled = TRUE scsi0:1.ctkEnabled = TRUE scsi0:2.ctkEnabled = TRUE ... ...- Power on the VM.
Verification
-
In the vSphere Client, check the VM configuration for the
changeTrackingEnabledparameter.
1.4. Migration tools and workflows Copy linkLink copied to clipboard!
You can use the Red Hat OpenStack VMware Migration toolkit Ansible collection to deploy an OpenStack instance in the destination cloud as a conversion host and configure the prerequisites in the destination cloud. You can gather information about the source cloud by using the VMWare community collection.
The following are the different phases of migration:
- The discovery phase when you analyze the VMWare source environment and collect data for the migration.
- The pre-migration phase when you make the destination cloud ready to accept the migration. You can configure your conversion host, the required network, and other configurations, as necessary.
- The migration phase.
To migrate your VMs, you can use the nbdkit server with a conversion host. You can configure the workflows with Ansible boolean variables. You can run the migration in a single cycle with minimal downtime if you use VMware’s change block tracking (CBT) option.
Figure 1.1. Migration of a VM from VMware to RHOSP with CBT
Figure 1.2. Migration of a VM from VMware to RHOSP with CBT continued
1.5. Migration features Copy linkLink copied to clipboard!
You can use the following features when migrating:
- Discovery mode.
- Network mapping.
- Port creation and MAC addresses mapping.
- Openstack flavor mapping and creation.
-
Migration with
nbdkitserver with change block tracking (CBT) feature. - Multi-disk migration.
- Multiple NICs.
- Parallel migration on the same conversion host.
- Ansible Automation Platform (AAP)
1.6. Supported operating systems for virt-v2v Copy linkLink copied to clipboard!
The VMware Migration Toolkit uses virt-v2v for conversion. For a list of supported guest operating systems for virt-v2v, see the Red Hat Knowledgebase article: Converting virtual machines from other hypervisors to KVM with virt-v2v in RHEL 7, RHEL 8, RHEL 9, and RHEL 10.
Red Hat Openstack Platform (RHOSP) uses Kernel-based Virtual Machine (KVM) for hypervisors. For a list of certified guest operating systems for KVM, see the Red Hat Knowledgebase article: Certified Guest Operating Systems in Red Hat OpenStack Platform, Red Hat Virtualization, Red Hat OpenShift Virtualization and Red Hat Enterprise Linux with KVM.
Chapter 2. Deploying a conversion host Copy linkLink copied to clipboard!
Deploy an Red Hat OpenStack Platform (RHOSP) instance in the destination cloud as a conversion host.
2.1. Deploying a conversion host Copy linkLink copied to clipboard!
You can use a conversion host to assist in the migration process.
Prerequisites
- Tenant access to a Red Hat OpenStack Platform (RHOSP) environment.
- Red Hat Enterprise Linux (RHEL) 9.6 or later image uploaded into the Red Hat OpenStack Image service.
- A RHOSP flavor with a recommended minimum 6 vCPU, 8GB RAM and 20GB disk.
- VMware SDK.
Procedure
- Create a conversion host instance by following the instructions in Creating an instance in the Creating and managing instances guide.
- Attach a floating IP to the conversion host instance following the instructions in Assigning a specific floating IP in the Managing networking resources guide.
-
Log in to the conversion host as a
cloud-userwith the SSH key you created when you created the conversion host. Register your system either with the Red Hat Content Delivery Network (CDN) or with Red Hat Satellite. To register with CDN:
[cloud-user@conversion_host ~]$ sudo subscription-manager register- Install the VMware SDK library and ensure that you have root ownership.
Chapter 3. Configuring automation execution environment images Copy linkLink copied to clipboard!
Automation execution environment is a container image that provides a standardized runtime environment for Ansible automation to ensure consistent behavior across different environments and to simplify deployment and maintenance. You can use containerized automation execution environment images that encapsulate the dependencies to run migration playbooks in a consistent, isolated environment.
The Red Hat OpenStack VMware Migration toolkit AEE image contains:
- Ansible Core.
- Red Hat OpenStack VMware Migration toolkit Ansible collection.
- The dependencies for VMware to Red Hat OpenStack Platform (RHOSP) migration.
3.1. Building automation execution environment images Copy linkLink copied to clipboard!
To ensure that you use stable, consistent, reproducible builds with the latest updates, you can create automation execution environment images by creating a virtual environment and installing the dependencies.
Prerequisites
-
The
ansible-buildertool for building execution environments is installed. For more information about installingansible-builder, see: Installing Ansible Builder -
The
podmanordockercommand-line tool for the container runtime is installed. - Authentication to registry.redhat.io
- Offline token in https://console.redhat.com/ansible/automation-hub/token/.
Procedure
Create the main execution environment configuration file:
$ cat <<EOF> execution-environment.yml --- version: 3 images: base_image: name: registry.redhat.io/ansible-automation-platform-25/ee-minimal-rhel9:latest options: package_manager_path: /usr/bin/microdnf dependencies: ansible_runner: package_pip: ansible-runner ansible_core: package_pip: ansible-core python: requirements.txt system: binddep.txt galaxy: requirements.yml python_interpreter: package_system: "python3.11" python_path: "/usr/bin/python3.11" additional_build_files: - src: ansible.cfg dest: . additional_build_steps: prepend_base: - "RUN mkdir -p /etc/sudoers.d" - "RUN echo 'cloud-user ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/cloud-user" EOFCreate the Python dependencies file:
$ cat << EOF > requirements.txt requests pyVim pyVmomi EOFCreate the Ansible collections requirements file:
$ cat << EOF > requirements.yml collections: - name: vmware.vmware version: 2.4.0 - name: vmware.vmware_rest version: 4.9.0 - name: os_migrate.vmware_migration_kit version: <collection-version> EOFwhere:
<collection-version>- Specifies the version number that you want to use, such as "2.2.3".
Create the system package dependencies file:
$ cat << EOF > binddep.txt openssh-clients sshpass python3 python3-pip python3-dnf rsync gcc python3-devel git EOFCreate the ansible.cfg configuration file using your offline token file:
$ cat << EOF > ansible.cfg [galaxy] server_list = automation_hub [galaxy_server.automation_hub] url=https://console.redhat.com/api/automation-hub/content/published/ auth_url=https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token token=<your_console_redhat_com_token> EOFLog in to
podmanto authenticate:$ podman login registry.redhat.ioBuild the AEE image:
$ ansible-builder build --tag vmware-migration-kit:latest- Publish your execution environment to Ansible Automation Platform or any other registry by following the instructions in Publishing an automation execution environment in the Creating and using execution environments Red Hat Ansible Automation Platform guide.
Chapter 4. Configuring AAP to launch a migration Copy linkLink copied to clipboard!
Configure Ansible Automation Platform (AAP) to launch a migration.
Prerequisites
- You have deployed your conversion host.
- You have the IP address and hostname for your conversion host.
- You have the user, public, and private key SSH access to the conversion host.
4.1. Ansible YAML files for migration Copy linkLink copied to clipboard!
Prepare the Ansible YAML files the you use with the Ansible job template.
myvars.yml
# if you run the migration from an Ansible Execution Environment (AEE)
# set this to true:
runner_from_aee: true
# osm working directory:
os_migrate_vmw_data_dir: /opt/os-migrate
copy_openstack_credentials_to_conv_host: false
# Re-use an already deployed conversion host:
already_deploy_conversion_host: true
# If no mapped network then set the openstack network:
openstack_private_network: 81cc01d2-5e47-4fad-b387-32686ec71fa4
# Security groups for the instance:
security_groups: ab7e2b1a-b9d3-4d31-9d2a-bab63f823243
use_existing_flavor: true
# key pair name, could be left blank
ssh_key_name: default
# network settings for openstack:
os_migrate_create_network_port: true
copy_metadata_to_conv_host: true
used_mapped_networks: false
vms_list:
- VM1
- VM2
- My_VM
- Windows_VM
- RHEL9_VM
- Accounting_VM
- ...
secrets.yml
# VMware parameters:
esxi_hostname: 10.0.0.7
vcenter_hostname: 10.0.0.7
vcenter_username: root
vcenter_password: root
vcenter_datacenter: Datacenter
os_cloud_environ: psi-rhos-upgrades-ci
dst_cloud:
auth:
auth_url: https://keystone-public-openstack.apps.ocp-4-16.standalone
username: admin
project_id: xyz
project_name: admin
user_domain_name: Default
password: openstack
region_name: regionOne
interface: public
insecure: true
identity_api_version: 3
4.2. Creating credentials for AAP Copy linkLink copied to clipboard!
Create credentials in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Infrastructure,Credentials. -
Click
Create Credentials. Set the following parameters:
Name: Bastion key Credential Type: Machine Username: cloud-user SSH Private Key: key image-
Click
Create Credential.
4.3. Creating an inventory for AAP Copy linkLink copied to clipboard!
Create an inventory in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Infrastructure,Inventories. -
Click
Create Inventory,Create Inventory. Set the following parameters:
Name: Conversion Host Inventory Organization: Default-
Click
Create Inventory.
4.4. Creating hosts for AAP Copy linkLink copied to clipboard!
Create hosts in Ansible Automation Platform (AAP).
Procedure
Create the conversion host:
-
From the navigation panel, select
Automation Execution,Infrastructure,Hosts. -
Click
Create Host. Set the following parameters:
Name: conversion_host Inventory: Conversion Host Inventory Variables: ansible_ssh_user: cloud-user ansible_host: <IP_or_hostname>1 - 1
- Replace
<IP_or_hostname>with the IP address or hostname of your Ansible host.
-
Click
Create Host.
-
From the navigation panel, select
Create the migrator host.
-
From the navigation panel, select
Automation Execution,Infrastructure,Hosts. -
Click
Create Host. Set the following parameters:
Name: migrator Inventory: Conversion Host Inventory Variables: ansible_connection: local ansible_python_interpreter: '{{ ansible_playbook_python }}'-
Click
Create Host.
-
From the navigation panel, select
4.5. Creating an execution environment Copy linkLink copied to clipboard!
Create an execution environment in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Infrastructure,Execution Environments. -
Click
Create Execution Environment. Set the following parameters:
Name: VMware migration toolkit Execution Environment Image: <insert the image that pushed to a repository when you built automation execution environment image>-
Click
Create Execution Environment.
-
Click
4.6. Creating a project Copy linkLink copied to clipboard!
Create a project in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Projects. -
Click
Create Project Set the following parameters:
Name: VMware migration toolkit project Organization: Default Execution Environment: VMware migration toolkit Execution Environment Environment Source Control Type: Git Source Control URL: https://github.com/os-migrate/vmware-migration-kit tag:"v2.1.0"-
Click
Create Project.
4.7. Configuring the job template Copy linkLink copied to clipboard!
Configure the job template in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Templates. -
Click
Create Template,Create Job Template Set the following parameters:
Name: Migration Inventory: Conversion Host Inventory Project: VMware migration toolkit project Playbook: playbooks/migration.yml Execution Environment: vmware migration toolkit Execution Environment Credentials: Bastion key Extra Variables: <Add your extra variables here, such as myvars.yml and secrets.yml>-
Click
Create Job Template.
4.8. Launching the migration using execution Copy linkLink copied to clipboard!
Launch the migration in Ansible Automation Platform (AAP).
Procedure
-
From the navigation panel, select
Automation Execution,Templates. -
Locate the
Migration Jobtemplate. - Click the rocket icon to launch the migration.
Chapter 5. Migration performance expectations Copy linkLink copied to clipboard!
Your migration performance is dependent on your environment and your application of the best practice guidelines.
5.1. Migration best practices Copy linkLink copied to clipboard!
The following are some migration best practices, when you migrate a large workload from VMware to Red Hat Openstack Platform (RHOSP) with Ansible Automation Platform (AAP) using a conversion host:
- Ensure that you size your destination RHOSP environment appropriately to accommodate large numbers of instances, ports, volumes, floating IPs, API requests, and other RHOSP resources.
- Divide your workload among multiple conversion hosts and threads for faster migration.
A conversion host typically contains the following configuration:
- 6 vCPU.
- 8GB RAM.
- 20GB disk.
Red Hat Enterprise Linux 9.5 or later.
- Known issues
- If a single conversion host performs many 1,000s of migrations in a short time, there might be issues with the device mount mechanism in the operating system. You can remedy this by rebooting the conversion host to clean the /dev/ directory.
-
If you encounter the message
snapshot hierarchy is too deep, you can clean the guest snapshots hierarchy and re-run the migration.
5.2. Examples of migration times Copy linkLink copied to clipboard!
The following tables contain example performance measurements for migrating from VMware to Red Hat Openstack Platform (RHOSP) with Ansible Automation Platform (AAP).
| VMs | Threads | Time |
|---|---|---|
| 1 | 1 | 2 minutes |
| 20 | 1 | 31 minutes |
| 20 | 2 | 15 minutes |
| 30 | 10 | 8 minutes |
| 100 | 10 | 20 minutes |
| Disk size | CBT enabled | Time | Cutover time | Expected downtime |
|---|---|---|---|---|
| 100 GB | yes | 8 minutes | 2 minutes | 2 minutes |
| 100 GB | no | 7 minutes | - | 7 minutes |
| 200 GB | yes | 10 minutes 30 seconds | 2 minutes | 2 minutes |
| 200 GB | no | 9 minutes 30 seconds | - | 9 minutes 30 seconds |
| 300 GB | yes | 17 minutes | 2 minutes | 2 minutes |
| 300 GB | no | 15 minutes | - | 15 minutes |
| 1 TB | yes | 39 minutes | 2 minutes | 2 minutes |
| 1 TB | no | 35 minutes | - | 35 minutes |
Enabling Change Block Tracking (CBT) takes more time overall to migrate but with less downtime because of the smaller data cutover time.
| Threads | CBT enabled | Time | Synch time |
|---|---|---|---|
| 1 | yes | 115 minutes | |
| 5 | no | 105 minutes | |
| 5 | no | 22 minutes | 110 minutes |
Parallelizing the migration on a single conversion host with 5 threads and enabling CBT to pre-migrate the volume data reduces the downtime.
| VMs | Conversion hosts | Threads | Time |
|---|---|---|---|
| 250 | 2 | 6 | 60 minutes |
| 1000 | 2 | 6 | 5 hours |
| 1500 | 2 | 6 | > 5 hours |
The conversion host flavor has 12 GB of RAM and 6 vCPUs, and can run 6 migrations in parallel.
Chapter 6. Troubleshooting VM migration Copy linkLink copied to clipboard!
In case you encounter some of the following issues when you migrate VMs, you can use the included mitigation steps.
6.1. Connectivity from conversion host to source environment Copy linkLink copied to clipboard!
In case you experience network connectivity issues between your conversion host and your source VMware environment, you can ensure that the network and name resolution are properly configured before running migrations.
Port 902 must be reachable from the conversion host. You can test the connectivity using the
curlcommand:$ curl -v telnet://<vcenter_ip>:902You can test the connectivity using the
Netcatcommand:$ nc -zv <vcenter_ip> 902If the vCenter FQDN hostname fails to resolve from the conversion host, you can update the
/etc/hostsdirectory:$ echo "<vcenter_ip> vcenter.domain.local" | sudo tee -a /etc/hosts
Replace <vcenter_ip> with the IP address of the vCenter that you want to connect to.
6.2. Unreachable metadata service Copy linkLink copied to clipboard!
If the metadata service is not reachable, you might see errors such as the following:
Failed to fetch metadata: Get "http://169.254.169.254/openstack/latest/meta_data.json": dial tcp 169.254.169.254:80: connect: no route to host
As a workaround, you can set a manual instance UUID in the import playbook:
import_workloads_instance_uuid: <uuid>
Replace <uuid> with your required UUID.
6.3. Enable debugging flags during migration Copy linkLink copied to clipboard!
You can increase the verbosity of the logs to assist your troubleshooting, by setting the following parameter:
import_workloads_debug: true
When you migrate successfully, the migration log file is located on the conversion host in the /tmp directory. If the migration failed, the log file is stored in the /opt/os-migrate directory. The name of the log file is usually in the format osm-nbdkit-<vm-name>-<random-id>.log, where <vm-name> is the name of the VM that was logged and <random-id> is a string generated for the migration.
6.4. NBDKit errors Copy linkLink copied to clipboard!
You might encounter an NBDKit error, such as the following:
nbdkit: error: server has no export named '': No such file or directory
The following are potential causes for this NBDKit error:
- Port 902 is not open between the conversion host and vCenter.
- The vCenter FQDN is not resolvable.
-
The
nbdkitcommand might have invalid characters or parameters.
6.5. Debugging manually Copy linkLink copied to clipboard!
You can use the following commands for manual troubleshooting.
To run NDBKit manually, run the command shown in the logs with the
verboseoption and enclose the VMDK path in double quotes:$ nbdkit --verbose vddk ".../guest-00001.vmdk"
If the migration snapshot has been deleted, remove the snapshot option, and use the base disk instead.
To run NBDcopy in another shell, run the nbdcopy command as shown in the logs and observe the NBDKit output. The expected output is:
vddk: config_complete.-
You can analyze the authentication and the paths of the migration process. The VMDK path is returned by the VMware API, typically:
[Datastore 1] path/to/the/guest-00001.vmdk.