Configuring and managing Windows virtual machines

Next steps

• To start using virtualization in RHEL 10, see Preparing RHEL to host virtual machines.
• In addition to RHEL 10 virtualization, Red Hat offers a number of specialized virtualization solutions, each with a different user focus and features. For more information, see Red Hat virtualization solutions.

• Flexible and fine-grained allocation of resources

  A VM runs on a host machine, which is usually physical, and physical hardware can also be assigned for the guest OS to use. However, the allocation of physical resources to the VM is done on the software level, and is therefore very flexible. A VM uses a configurable fraction of the host memory, CPUs, or storage space, and that configuration can specify very fine-grained resource requests.

  For example, what the guest OS sees as its disk can be represented as a file on the host file system, and the size of that disk is less constrained than the available sizes for physical disks.

• Software-controlled configurations

  The entire configuration of a VM is saved as data on the host, and is under software control. Therefore, a VM can easily be created, removed, cloned, migrated, operated remotely, or connected to remote storage.

  In addition, the current state of the VM can be backed up as a snapshot at any time. A snapshot can then be loaded to restore the system to the saved state.

• Separation from the host

  A guest OS runs on a virtualized kernel, separate from the host OS. This means that any OS can be installed on a VM, and even if the guest OS becomes unstable or is compromised, the host is not affected in any way.

• Space and cost efficiency

  A single physical machine can host a large number of VMs. Therefore, it avoids the need for multiple physical machines to do the same tasks, and thus lowers the space, power, and maintenance requirements associated with physical hardware.

• Software compatibility

  Because a VM can use a different OS than its host, virtualization makes it possible to run applications that were not originally released for your host OS. For example, using a RHEL 8 guest OS, you can run applications released for RHEL 8 on a RHEL 10 host system.

• Metadata such as the name of the VM, time zone, and other information about the VM.
• A description of the devices in the VM, including virtual CPUs (vCPUS), storage devices, input/output devices, network interface cards, and other hardware, real and virtual.
• VM settings such as the maximum amount of memory it can use, restart settings, and other settings about the behavior of the VM.

• virsh - A versatile virtualization command-line utility and shell with a great variety of purposes, depending on the provided arguments. For example:

  • Starting and shutting down a VM - virsh start and virsh shutdown
  • Listing available VMs - virsh list
  • Creating a VM from a configuration file - virsh create
  • Entering a virtualization shell - virsh

  For more information, see the virsh(1) man page on your system.

• virt-install - A CLI utility for creating new VMs. For more information, see the virt-install(1) man page on your system.
• virt-xml - A utility for editing the configuration of a VM.
• guestfish - A utility for examining and modifying VM disk images. For more information, see the guestfish(1) man page on your system.

• The RHEL 10 web console, also known as Cockpit, provides a remotely accessible and easy to use graphical user interface for managing VMs and virtualization hosts.

• Advanced networking - You cannot set up system bridges or tap devices. You are limited to user-mode (passt) networking, and cannot configure full external visibility of the VM.
• PCI device passthrough - Modifying the device assignment of PCI host hardware for the VM is not possible.
• Autostart - VMs in the session connection cannot automatically start on system boot.
• System-level storage pools and VM logs - In the system connection, storage pools and VM log files are saved in system directories, such as /etc/libvirt and /var/lib/libvirt. In the session connection, the user is limited to files saved in their home directory. This prevents managing host-wide storage or viewing logs centrally.

Prerequisites

• Red Hat Enterprise Linux 10 is installed and registered on your host machine.

• Your system meets the following hardware requirements to work as a virtualization host:

  • The following minimum system resources are available:

    • 6 GB free disk space for the host, plus another 6 GB for each intended VM.
    • 2 GB of RAM for the host, plus another 2 GB for each intended VM.

Troubleshooting

• If KVM virtualization is not supported by your host CPU, virt-host-validate generates the following output:

  QEMU: Checking for hardware virtualization: FAIL (Only emulated CPUs are available, performance will be significantly limited)

  Copy to Clipboard

  However, VMs on such a host system will fail to boot, rather than have performance problems.

  To work around this, you can change the <domain type> value in the XML configuration of the VM to qemu. Note, however, that Red Hat does not support VMs that use the qemu domain type, and setting this is highly discouraged in production environments.

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

Procedure

• Install the cockpit-machines plug-in.

  # dnf install cockpit-machines

  Copy to Clipboard

Prerequisites

• Virtualization is enabled on your host system.
• You have a sufficient amount of system resources to allocate to your VMs, such as disk space, RAM, or CPUs. The recommended values might vary significantly depending on the intended tasks and workload of the VMs.

• An operating system (OS) installation source is available locally or on a network. This can be one of the following:

  • An ISO image of an installation medium

  • A disk image of an existing VM installation

    Warning

    Installing from a host CD-ROM or DVD-ROM device is not possible in RHEL 10. If you select a CD-ROM or DVD-ROM as the installation source when using any VM installation method available in RHEL 10, the installation will fail. For more information, see the Red Hat Knowledgebase.

• To create a VM that uses the system connection of libvirt, you must have root privileges or be in the libvirt user group on the host. For more information, see User-space connection types for virtualization.
• Optional: A Kickstart file can be provided for faster and easier configuration of the installation.

• --name: the name of the new machine
• --memory: the amount of allocated memory
• --vcpus: the number of allocated virtual CPUs
• --disk: the type and size of the allocated storage
• --cdrom or --location: the type and location of the OS installation source

• --osinfo: the OS type and version that you intend to install

  Note

  To list all available values for the --osinfo argument, run the virt-install --osinfo list command.

  For more details, you can also run the osinfo-query os command. However, you might need to install the libosinfo-bin package first.

• Create a VM and install an OS from a local ISO file:

  • The following command creates a VM named demo-guest1 that installs the Windows 10 OS from an ISO image locally stored in the /home/username/Downloads/Win10install.iso file. This VM is also allocated with 2048 MiB of RAM and 2 vCPUs, and an 80 GiB qcow2 virtual disk is automatically configured for the VM.

    # virt-install \
        --name demo-guest1 --memory 2048 \
        --vcpus 2 --disk size=80 --osinfo win10 \
        --cdrom /home/username/Downloads/Win10install.iso

    Copy to Clipboard

• Create a VM, install an OS from a live CD, and do not create a permanent disk:

  • The following command creates a VM named demo-guest2 that uses the /home/username/Downloads/rhel10.iso image to run a RHEL 10 OS from a live CD. No disk space is assigned to this VM, so changes made during the session will not be preserved. In addition, the VM is allocated with 4096 MiB of RAM and 4 vCPUs.

    # virt-install \
        --name demo-guest2 --memory 4096 --vcpus 4 \
        --disk none --livecd --osinfo rhel10.0 \
        --cdrom /home/username/Downloads/rhel10.iso

    Copy to Clipboard

• Create a VM and import an existing disk image:

  • The following command creates a RHEL 10 VM named demo-guest3 that connects to an existing disk image, /home/username/backup/disk.qcow2. This is similar to physically moving a hard drive between machines, so the OS and data available to demo-guest3 are determined by how the image was handled previously. In addition, this VM is allocated with 2048 MiB of RAM and 2 vCPUs.

    # virt-install \
        --name demo-guest3 --memory 2048 --vcpus 2 \
        --osinfo rhel10.0 --import \
        --disk /home/username/backup/disk.qcow2

    Copy to Clipboard

    Note that you must use the --osinfo option when importing a disk image. If it is not provided, the performance of the created VM will be negatively affected.

• Create a VM and install an OS from a remote URL:

  • The following command creates a VM named demo-guest4 that installs from the http://example.com/OS-install URL. For the installation to start successfully, the URL must contain a working OS installation tree. In addition, the OS is automatically configured by using the /home/username/ks.cfg kickstart file. This VM is also allocated with 2048 MiB of RAM, 2 vCPUs, and a 160 GiB qcow2 virtual disk.

    # virt-install \
        --name demo-guest4 --memory 2048 --vcpus 2 --disk size=160 \
        --osinfo rhel10.0 --location http://example.com/OS-install \
        --initrd-inject /home/username/ks.cfg --extra-args="inst.ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8"

    Copy to Clipboard

• Create a VM and install an OS in a text-only mode:

  • The following command creates a VM named demo-guest5 that installs from a RHEL10.iso image file in text-only mode, without graphics. It connects the guest console to the serial console. The VM has 16384 MiB of memory, 16 vCPUs, and 280 GiB disk. This kind of installation is useful when connecting to a host over a slow network link.

    # virt-install \
        --name demo-guest5 --memory 16384 --vcpus 16 --disk size=280 \
        --osinfo rhel10.0 --location RHEL10.iso \
        --graphics none --extra-args='console=ttyS0'

    Copy to Clipboard

• Create a VM on a remote host:

  • The following command creates a VM named demo-guest6, which has the same configuration as demo-guest5, but resides on the 192.0.2.1 remote host.

    # virt-install \
        --connect qemu+ssh://root@192.0.2.1/system --name demo-guest6 --memory 16384 \
        --vcpus 16 --disk size=280 --osinfo rhel10.0 --location RHEL10.iso \
        --graphics none --extra-args='console=ttyS0'

    Copy to Clipboard

• Create a VM on a remote host and use a DASD mediated device as storage:

  • The following command creates a VM named demo-guest-7, which has the same configuration as demo-guest5, but for its storage, it uses a DASD mediated device mdev_30820a6f_b1a5_4503_91ca_0c10ba12345a_0_0_29a8, and assigns it device number 1111.

    # virt-install \
        --name demo-guest7 --memory 16384 --vcpus 16 --disk size=280 \
        --osinfo rhel10.0 --location RHEL10.iso --graphics none \
        --disk none --hostdev mdev_30820a6f_b1a5_4503_91ca_0c10ba12345a_0_0_29a8,address.type=ccw,address.cssid=0xfe,address.ssid=0x0,address.devno=0x1111,boot-order=1 \
        --extra-args 'rd.dasd=0.0.1111'

    Copy to Clipboard

    Note that the name of the mediated device available for installation can be retrieved by using the virsh nodedev-list --cap mdev command.

Verification

• If the VM is created successfully, a virt-viewer window opens with a graphical console of the VM and starts the guest OS installation.

Troubleshooting

• If virt-install fails with a cannot find default network error:

  • Ensure that the libvirt-daemon-config-network package is installed:

    # dnf info libvirt-daemon-config-network
    Installed Packages
    Name         : libvirt-daemon-config-network
    [...]

    Copy to Clipboard

  • Verify that the libvirt default network is active and configured to start automatically:

    # virsh net-list --all
     Name      State    Autostart   Persistent
    --------------------------------------------
     default   active   yes         yes

    Copy to Clipboard

  • If it is not, activate the default network and set it to auto-start:

    # virsh net-autostart default
    Network default marked as autostarted
    
    # virsh net-start default
    Network default started

    Copy to Clipboard

    • If activating the default network fails with the following error, the libvirt-daemon-config-network package has not been installed correctly.

      error: failed to get network 'default'
      error: Network not found: no network with matching name 'default'

      Copy to Clipboard

      To fix this, re-install libvirt-daemon-config-network:

      # dnf reinstall libvirt-daemon-config-network

      Copy to Clipboard

    • If activating the default network fails with an error similar to the following, a conflict has occurred between the default network’s subnet and an existing interface on the host.

      error: Failed to start network default
      error: internal error: Network is already in use by interface ens2

      Copy to Clipboard

      To fix this, use the virsh net-edit default command and change the 192.0.2.* values in the configuration to a subnet not already in use on the host.

Prerequisites

• A Windows OS installation source, which can be one of the following, and be available locally or on a network:

  • An ISO image of an installation medium
  • A disk image of an existing VM installation

• A storage medium with the KVM virtio drivers.

  To create this medium, see Preparing virtio driver installation media on a host machine.

• If you are installing Windows 11, the edk2-ovmf, swtpm and libtpms packages must be installed on the host.

Prerequisites

• The NetKVM driver is installed on the virtual machine.

  For more information, see Installing KVM paravirtualized drivers for Windows virtual machines.

• Remove unused devices, such as USBs or CD-ROMs, and disable the ports.

• Disable background services, such as SuperFetch and Windows Search. For more information about stopping services, see Disabling system services or Stop-Service.

• Disable useplatformclock. To do so, run the following command,

  # bcdedit /set useplatformclock No

  Copy to Clipboard
• Review and disable unnecessary scheduled tasks, such as scheduled disk defragmentation. For more information about how to do so, see Disable Scheduled Tasks.
• Make sure the disks are not encrypted.
• Reduce periodic activity of server applications. You can do so by editing the respective timers. For more information, see Multimedia Timers.
• Close the Server Manager application on the VM.
• Disable the antivirus software. Note that disabling the antivirus might compromise the security of the VM.
• Disable the screen saver.
• Keep the Windows OS on the sign-in screen when not in use.

Prerequisites

• An inactive VM that is already defined.
• The name of the VM.

• For remote VMs:

  • The IP address of the host where the VM is located.
  • Root access privileges to the host.
• If you want to start a VM that uses the system connection of libvirt, you must have root privileges or be in the libvirt user group on the host. For more information, see User-space connection types for virtualization.

Procedure

• For a local VM, use the virsh start utility.

  For example, the following command starts the demo-guest1 VM.

  # virsh start demo-guest1
  Domain 'demo-guest1' started

  Copy to Clipboard

• For a VM located on a remote host, use the virsh start utility along with the QEMU+SSH connection to the host.

  For example, the following command starts the demo-guest1 VM on the 192.0.2.1 host.

  # virsh -c qemu+ssh://root@192.0.2.1/system start demo-guest1
  
  root@192.0.2.1's password:
  
  Domain 'demo-guest1' started

  Copy to Clipboard

• Starting virtual machines automatically when the host starts

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.
• An inactive VM that is already defined.
• The name of the VM.
• If you want to start a VM that uses the system connection of libvirt, you must have root privileges or be in the libvirt user group on the host. For more information, see User-space connection types for virtualization.

Prerequisites

• A created virtual machine

Prerequisites

• Your system, as well as the VM you are connecting to, must support graphical displays.
• If the target VM is located on a remote host, connection and root access privileges to the host are needed.

Procedure

• To connect to a local VM, use the following command and replace guest-name with the name of the VM you want to connect to:

  # virt-viewer guest-name

  Copy to Clipboard

• To connect to a remote VM, use the virt-viewer command with the SSH protocol. For example, the following command connects as root to a VM called guest-name, located on remote system 192.0.2.1. The connection also requires root authentication for 192.0.2.1.

  # virt-viewer --direct --connect qemu+ssh://root@192.0.2.1/system guest-name
  root@192.0.2.1's password:

  Copy to Clipboard

Troubleshooting

• If clicking in the graphical console does not have any effect, expand the console to full screen. This is a known issue with the mouse cursor offset.

Prerequisites

• You have network connection and root access privileges to the target VM.
• If the target VM is located on a remote host, you also have connection and root access privileges to that host.

• Your VM network assigns IP addresses by dnsmasq generated by libvirt. This is the case for example in libvirt NAT networks.

  Notably, if your VM is using one of the following network configurations, you cannot connect to the VM by using SSH:

  • hostdev interfaces
  • Direct interfaces
  • Bridge interaces

• The libvirt-nss component is installed and enabled on the VM’s host. If it is not, do the following:

  1. Install the libvirt-nss package:

     # dnf install libvirt-nss

     Copy to Clipboard

  2. Edit the /etc/nsswitch.conf file and add libvirt_guest to the hosts line:

     ...
     passwd:      compat
     shadow:      compat
     group:       compat
     hosts:       files libvirt_guest dns
     ...

     Copy to Clipboard

Troubleshooting

• If you do not know the VM’s name, you can list all VMs available on the host by using the virsh list --all command:

  # virsh list --all
  Id    Name                           State
  ----------------------------------------------------
  2     testguest1                    running
  -     testguest2                    shut off

  Copy to Clipboard

• Does not have a network connection, and thus cannot be interacted with using SSH.

Prerequisites

• The GRUB boot loader on your host must be configured to use serial console. To verify, check that the /etc/default/grub file on your host contains the GRUB_TERMINAL=serial parameter.

  $ sudo grep GRUB_TERMINAL /etc/default/grub
  GRUB_TERMINAL=serial

  Copy to Clipboard

• The VM must have a serial console device configured, such as console type='pty'. To verify, do the following:

  # virsh dumpxml vm-name | grep console
  
  <console type='pty' tty='/dev/pts/2'>
  </console>

  Copy to Clipboard

• The VM must have the serial console configured in its kernel command line. To verify this, the cat /proc/cmdline command output on the VM should include console=<console-name>, where <console-name> is architecture-specific:

  • For AMD64 and Intel 64: ttyS0

    Note

    The following commands in this procedure use ttyS0.

    # cat /proc/cmdline
    BOOT_IMAGE=/vmlinuz-6.12.0-0.el10_0.x86_64 root=/dev/mapper/rhel-root ro console=tty0 console=ttyS0,9600n8 rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb

    Copy to Clipboard

    If the serial console is not set up properly on a VM, using virsh console to connect to the VM connects you to an unresponsive guest console. However, you can still exit the unresponsive console by using the Ctrl+] shortcut.

  • To set up serial console on the VM, do the following:

    1. On the VM, enable the console=ttyS0 kernel option:

       # grubby --update-kernel=ALL --args="console=ttyS0"

       Copy to Clipboard

    2. Clear the kernel options that might prevent your changes from taking effect.

       # grub2-editenv - unset kernelopts

       Copy to Clipboard
    3. Reboot the VM.

• The serial-getty@<console-name> service must be enabled. For example, on AMD64 and Intel 64:

  # systemctl status serial-getty@ttyS0.service
  
  ○ serial-getty@ttyS0.service - Serial Getty on ttyS0
       Loaded: loaded (/usr/lib/systemd/system/serial-getty@.service; enabled; preset: enabled)

  Copy to Clipboard

• You can edit the /etc/libvirt/libvirt.conf file and set the value of the uri_default parameter to example-qemu-host-alias as a default libvirt target.

  # These can be used in cases when no URI is supplied by the application
  # (@uri_default also prevents probing of the hypervisor driver).
  #
  uri_default = "example-qemu-host-alias"

  Copy to Clipboard

  As a result, all libvirt-based commands will automatically be performed on the specified remote host.

  $ virsh list
  root@192.0.2.1's password:
  
  Id   Name              State
  ---------------------------------
  1   example-remote-guest      running

  Copy to Clipboard

• The -c (or --connect) option can be used to run the virt-install,

Shutting down a responsive VM

• If you are connected to the guest, use a shutdown command or a GUI element appropriate to the guest operating system.

  Note

  In some environments, such as in Linux guests that use the GNOME Desktop, using the GUI power button for suspending or hibernating the guest might instead shut down the VM.

• Alternatively, use the virsh shutdown command on the host:

  • If the VM is on a local host:

    # virsh shutdown demo-guest1
    Domain 'demo-guest1' is being shutdown

    Copy to Clipboard

  • If the VM is on a remote host, in this example 192.0.2.1:

    # virsh -c qemu+ssh://root@192.0.2.1/system shutdown demo-guest1
    
    root@192.0.2.1's password:
    Domain 'demo-guest1' is being shutdown

    Copy to Clipboard

Verification

• On the host, display the list of your VMs to see their status.

  # virsh list --all
  
   Id    Name                 State
  ------------------------------------------
   1     demo-guest1          shut off

  Copy to Clipboard

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.

Troubleshooting

• If the VM does not shut down, click the Menu button ⋮ next to the Shut Down button and select Force Shut Down.
• To shut down an unresponsive VM, you can also send a non-maskable interrupt by clicking the Send non-maskable interrupt button in the Menu.

Restarting a responsive VM

• If you are connected to the guest, use a restart command or a GUI element appropriate to the guest operating system.

• Alternatively, use the virsh reboot command on the host:

  • If the VM is on a local host:

    # virsh reboot demo-guest1
    Domain 'demo-guest1' is being rebooted

    Copy to Clipboard

  • If the VM is on a remote host, in this example 192.0.2.1:

    # virsh -c qemu+ssh://root@192.0.2.1/system reboot demo-guest1
    
    root@192.0.2.1's password:
    Domain 'demo-guest1' is being rebooted

    Copy to Clipboard

Verification

• On the host, display the list of your VMs to see their status.

  # virsh list --all
  
   Id    Name                 State
  ------------------------------------------
   1     demo-guest1          running

  Copy to Clipboard

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.

Troubleshooting

• If the VM does not restart, click the Menu button ⋮ next to the Reboot button and select Force Reboot.
• To shut down an unresponsive VM, you can also send a non-maskable interrupt by clicking the Send non-maskable interrupt button in the Menu.

Prerequisites

• Back up important data from the VM.
• Shut down the VM.
• Make sure no other VMs use the same associated storage.

Procedure

• Use the virsh undefine utility.

  For example, the following command removes the guest1 VM, its associated storage volumes, and non-volatile RAM, if any.

  # virsh undefine guest1 --remove-all-storage --nvram
  Domain 'guest1' has been undefined
  Volume 'vda'(/home/images/guest1.qcow2) removed.

  Copy to Clipboard

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.
• Back up important data from the VM.
• Make sure no other VM uses the same associated storage.
• Optional: Shut down the VM.

Procedure

• To obtain a list of VMs on your host:

  # virsh list --all
  Id   Name              State
  ----------------------------------
  1    testguest1             running
  -    testguest2             shut off
  -    testguest3             shut off
  -    testguest4             shut off

  Copy to Clipboard

• To obtain basic information about a specific VM:

  # virsh dominfo _testguest1
  Id:             1
  Name:           testguest1
  UUID:           a973666f-2f6e-415a-8949-75a7a98569e1
  OS Type:        hvm
  State:          running
  CPU(s):         2
  CPU time:       188.3s
  Max memory:     4194304 KiB
  Used memory:    4194304 KiB
  Persistent:     yes
  Autostart:      disable
  Managed save:   no
  Security model: selinux
  Security DOI:   0
  Security label: system_u:system_r:svirt_t:s0:c486,c538 (enforcing)

  Copy to Clipboard

• To obtain the complete XML configuration of a specific VM:

  # virsh dumpxml testguest2
  
  <domain type='kvm' id='1'>
    <name>testguest2</name>
    <uuid>a973434f-2f6e-4ěša-8949-76a7a98569e1</uuid>
    <metadata>
  [...]

  Copy to Clipboard

• For information about a VM’s disks and other block devices:

  # virsh domblklist testguest3
   Target   Source
  ---------------------------------------------------------------
   vda      /var/lib/libvirt/images/testguest3.qcow2
   sda      -
   sdb      /home/username/Downloads/virt-p2v-1.36.10-1.el7.iso

  Copy to Clipboard

• To obtain information about a VM’s file systems and their mountpoints:

  # virsh domfsinfo testguest3
  Mountpoint   Name   Type   Target
  ------------------------------------
   /            dm-0   xfs   vda
   /boot        vda2   xfs   vda
   /boot/efi    vda1   vfat  vda

  Copy to Clipboard

• To obtain more details about the vCPUs of a specific VM:

  # virsh vcpuinfo testguest4
  VCPU:           0
  CPU:            3
  State:          running
  CPU time:       103.1s
  CPU Affinity:   yyyy
  
  VCPU:           1
  CPU:            0
  State:          running
  CPU time:       88.6s
  CPU Affinity:   yyyy

  Copy to Clipboard

• To list all network interfaces of a specific VM:

  # virsh domiflist testguest5
  Interface   Type      Source    Model    MAC
  -------------------------------------------------------------
   vnet0       network   default   virtio   52:54:00:ad:23:fd
   vnet1       bridge    br0       virtio   52:54:00:40:d4:9d

  Copy to Clipboard

Prerequisites

• The web console VM plug-in is installed on your system.

Procedure

• Click Virtual Machines in the web console’s side menu.

  A dialog box appears with information about the available storage pools, available networks, and the VMs to which the web console is connected.

• Storage Pools - The number of storage pools, active or inactive, that can be accessed by the web console and their state.
• Networks - The number of networks, active or inactive, that can be accessed by the web console and their state.
• Name - The name of the VM.
• Connection - The type of libvirt connection, system or session.
• State - The state of the VM.
• Resource usage - Memory and virtual CPU usage of the VM.
• Disks - Detailed information about disks assigned to the VM.

• Unique settings, such as persistent network MAC configuration, which can prevent the clones from working correctly.
• Sensitive data, such as SSH keys and password files.

Prerequisites

• The source VM is shut down.
• Ensure that there is sufficient disk space to store the cloned disk images.
• The virt-install package is installed on the host.
• Optional: When creating multiple VM clones, remove unique data and settings from the source VM to ensure the cloned VMs work properly. For instructions, see Creating virtual machine templates.

Procedure

• Use the virt-clone utility with options that are appropriate for your environment and use case.

  Sample use cases

  • The following command clones a local VM named example-VM-1 and creates the example-VM-1-clone VM. It also creates and allocates the example-VM-1-clone.qcow2 disk image in the same location as the disk image of the original VM, and with the same data:

    # virt-clone --original example-VM-1 --auto-clone
    Allocating 'example-VM-1-clone.qcow2'                            | 50.0 GB  00:05:37
    
    Clone 'example-VM-1-clone' created successfully.

    Copy to Clipboard

  • The following command clones a VM named example-VM-2, and creates a local VM named example-VM-3, which uses only two out of multiple disks of example-VM-2:

    # virt-clone --original example-VM-2 --name example-VM-3 --file /var/lib/libvirt/images/disk-1-example-VM-2.qcow2 --file /var/lib/libvirt/images/disk-2-example-VM-2.qcow2
    Allocating 'disk-1-example-VM-2.qcow2'                                      | 78.0 GB  00:05:37
    Allocating 'disk-2-example-VM-2.qcow2'                                      | 80.0 GB  00:05:37
    
    Clone 'example-VM-3' created successfully.

    Copy to Clipboard

  • To clone your VM to a different host, migrate the VM without undefining it on the local host. For example, the following commands clone the previously created example-VM-3 VM to the 192.0.2.1 remote system, including its local disks. Note that you require root privileges to run these commands for 192.0.2.1:

    # virsh migrate --offline --persistent example-VM-3 qemu+ssh://root@192.0.2.1/system
    root@192.0.2.1's password:
    
    # scp /var/lib/libvirt/images/<disk-1-example-VM-2-clone>.qcow2 root@192.0.2.1/<user@remote_host.com>://var/lib/libvirt/images/
    
    # scp /var/lib/libvirt/images/<disk-2-example-VM-2-clone>.qcow2 root@192.0.2.1/<user@remote_host.com>://var/lib/libvirt/images/

    Copy to Clipboard

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.
• Ensure that the VM you want to clone is shut down.

Verification

• Confirm whether the cloned VM is displayed in the list of VMs available on your host.

• Storage located on a shared network
• The --copy-storage-all parameter for the virsh migrate command, which copies disk image contents from the source to the destination over the network.
• Storage area network (SAN) logical units (LUNs).
• Ceph storage clusters

• VMs that use certain features and configurations will not work correctly if migrated, or the migration will fail. Such features include:

  • Device passthrough

• A migration between hosts that use Non-Uniform Memory Access (NUMA) pinning works only if the hosts have similar topology. However, the performance on running workloads might be negatively affected by the migration.
• Both the source and destination hosts use specific RHEL versions that are supported for VM migration, see Supported hosts for virtual machine migration

• The physical CPUs, both on the source VM and the destination VM, must be identical, otherwise the migration might fail. Any differences between the VMs in the following CPU related areas can cause problems with the migration:

  • CPU model

    • Migrating between an Intel 64 host and an AMD64 host is unsupported, even though they share the x86-64 instruction set.
    • For steps to ensure that a VM will work correctly after migrating to a host with a different CPU model, see Verifying host CPU compatibility for virtual machine migration.
  • Physical machine firmware versions and settings

Offline migration

• The following command migrates a shut-off example-VM VM from your local host to the system connection of the example-destination host by using an SSH tunnel.

  # virsh migrate --offline --persistent <example_VM> qemu+ssh://example-destination/system

  Copy to Clipboard

Multi-FD live migration

• You can use multiple parallel connections to the destination host during the live migration. This is also known as multiple file descriptors (multi-FD) migration. With multi-FD migration, you can speed up the migration by utilizing all of the available network bandwidth for the migration process.

  # virsh migrate --live --persistent --parallel --parallel-connections 4 <example_VM> qemu+ssh://<example-destination>/system

  Copy to Clipboard

  This example uses 4 multi-FD channels to migrate the <example_VM> VM. It is a good practice to use one channel for each 10 Gbps of available network bandwidth. The default value is 2 channels.

Live migration with an increased downtime limit

• To improve the reliability of a live migration, you can set the maxdowntime parameter, which specifies the maximum amount of time, in milliseconds, the VM can be paused during live migration. Setting a larger downtime can help to ensure the migration completes successfully.

  # virsh migrate-setmaxdowntime <example_VM> <time_interval_in_milliseconds>

  Copy to Clipboard

Post-copy migration

• If your VM has a large memory footprint, you can perform a post-copy migration, which transfers the source VM’s CPU state first and immediately starts the migrated VM on the destination host. The source VM’s memory pages are transferred after the migrated VM is already running on the destination host. Because of this, a post-copy migration can result in a smaller downtime of the migrated VM.

  However, the running VM on the destination host might try to access memory pages that have not yet been transferred, which causes a page fault. If too many page faults occur during the migration, the performance of the migrated VM can be severely degraded.

  Given the potential complications of a post-copy migration, it is usually better to use the following command that starts a standard live migration and switches to a post-copy migration if the live migration cannot be finished in a specified amount of time.

  # virsh migrate --live --persistent --postcopy --timeout <time_interval_in_seconds> --timeout-postcopy <example_VM> qemu+ssh://<example-destination>/system

  Copy to Clipboard

Auto-converged live migration

• If your VM is under a heavy memory workload, you can use the --auto-converge option. This option automatically slows down the execution speed of the VM’s CPU. As a consequence, this CPU throttling can help to slow down memory writes, which means the live migration might succeed even in VMs with a heavy memory workload.

  However, the CPU throttling does not help to resolve workloads where memory writes are not directly related to CPU execution speed, and it can negatively impact the performance of the VM during a live migration.

  # virsh migrate --live --persistent --auto-converge <example_VM> qemu+ssh://<example-destination>/system

  Copy to Clipboard

Verification

• For offline migration:

  • On the destination host, list the available VMs to verify that the VM was migrated successfully.

    # virsh list --all
    Id      Name             State
    ----------------------------------
    10    example-VM-1      shut off

    Copy to Clipboard

• For live migration:

  • On the destination host, list the available VMs to verify the state of the destination VM:

    # virsh list --all
    Id      Name             State
    ----------------------------------
    10    example-VM-1      running

    Copy to Clipboard

    If the state of the VM is listed as running, it means that the migration is finished. However, if the live migration is still in progress, the state of the destination VM will be listed as paused.

• For post-copy migration:

  1. On the source host, list the available VMs to verify the state of the source VM.

     # virsh list --all
     Id      Name             State
     ----------------------------------
     10    example-VM-1      shut off

     Copy to Clipboard

  2. On the destination host, list the available VMs to verify the state of the destination VM.

     # virsh list --all
     Id      Name             State
     ----------------------------------
     10    example-VM-1      running

     Copy to Clipboard

     If the state of the source VM is listed as shut off and the state of the destination VM is listed as running, it means that the migration is finished.

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plugin is installed on your system.

• Hypervisor: The source host and the destination host both use the KVM hypervisor.
• Hosts: The source and destination hosts are running.

• Open ports: Ensure the following ports are open on the destination host.

  • Port 22 is needed for connecting to the destination host by using SSH.
  • Port 16514 is needed for connecting to the destination host by using TLS.
  • Port 16509 is needed for connecting to the destination host by using TCP.
  • Ports 49152-49215 are needed by QEMU for transfering the memory and disk migration data.
• CPU: The VM must be compatible with the CPU features of the destination host. To ensure this is the case, see Verifying host CPU compatibility for virtual machine migration.

• Storage: The disk images of VMs that will be migrated are accessible to both the source host and the destination host. This is optional for offline migration, but required for migrating a running VM. To ensure storage accessibility for both hosts, one of the following must apply:

  • You are using storage area network (SAN) logical units (LUNs).
  • You are using a Ceph storage clusters.
  • The disk image is located on a separate networked location. For instructions to set up such shared VM storage, see Sharing virtual machine disk images with other hosts.

• Network bandwidth: When migrating a running VM, your network bandwidth must be higher than the rate in which the VM generates dirty memory pages.

  To obtain the dirty page rate of your VM before you start the live migration, do the following on the command line:

  1. Monitor the rate of dirty page generation of the VM for a short period of time.

     # virsh domdirtyrate-calc vm-name 30

     Copy to Clipboard

  2. After the monitoring finishes, obtain its results:

     # virsh domstats vm-name --dirtyrate
     Domain: 'vm-name'
       dirtyrate.calc_status=2
       dirtyrate.calc_start_time=200942
       dirtyrate.calc_period=30
       dirtyrate.megabytes_per_second=2

     Copy to Clipboard

     In this example, the VM is generating 2 MB of dirty memory pages per second. Attempting to live-migrate such a VM on a network with a bandwidth of 2 MB/s or less will cause the live migration not to progress if you do not pause the VM or lower its workload.

     To ensure that the live migration finishes successfully, your network bandwidth should be significantly greater than the VM’s dirty page generation rate.

     Note

     The value of the calc_period option might differ based on the workload and dirty page rate. You can experiment with several calc_period values to determine the most suitable period that aligns with the dirty page rate in your environment.

• Bridge tap network specifics: When migrating an existing VM in a public bridge tap network, the source and destination hosts must be located on the same network. Otherwise, the VM network will not work after migration.

• Confirm whether the VM appears in the list of VMs available on the destination host.
• Start the migrated VM and observe if it boots up.

• Calculating dirty memory page rate generation of the VM.

  Currently, when migrating a VM with an attached Mellanox VF, live migration data and statistics provided by virsh domjobinfo and virsh domdirtyrate-calc commands are inaccurate, because the calculations only count guest RAM without including the impact of the attached VF.

• Using a post-copy live migration.
• Using a virtual I/O Memory Management Unit (vIOMMU) device in the VM.

• A CX-7 device with the same Parameter-Set Identification (PSID) and the same firmware version must be used on both the source and the destination hosts.

  You can check the PSID of your device with the following command:

  # mstflint -d <device_pci_address> query | grep -i PSID
  
  PSID: MT_1090111019

  Copy to Clipboard
• On one CX-7 physical function, you can use at maximum 4 VFs for live migration at the same time. For example, you can migrate one VM with 4 attached VFs, or 4 VMs with one VF attached to each VM.

Prerequisites

• You have a Mellanox CX-7 networking device with a firmware version that is equal to or greater than 28.36.1010.

  Refer to Mellanox documentation for details about supported firmware versions and ensure you are using an up-to-date version of the firmware.

• The host uses the Intel 64, AMD64, or ARM 64 CPU architecture.
• The Mellanox firmware version on the source host must be the same as on the destination host.

• The mstflint package is installed on both the source and destination host:

  # dnf install mstflint

  Copy to Clipboard

• The Mellanox CX-7 networking device has VF_MIGRATION_MODE set to MIGRATION_ENABLED:

  # mstconfig -d <device_pci_address> query | grep -i VF_migration
  
  VF_MIGRATION_MODE                           MIGRATION_ENABLED(2)

  Copy to Clipboard

  • You can set VF_MIGRATION_MODE to MIGRATION_ENABLED by using the following command:

    # mstconfig -d <device_pci_address> set VF_MIGRATION_MODE=2

    Copy to Clipboard

• The openvswitch package is installed on both the source and destination host:

  # dnf install openvswitch

  Copy to Clipboard
• All of the general SR-IOV devices prerequisites.

• All of the general VM migration prerequisites. For details, see Migrating a virtual machine by using the command line

Prerequisites

• You have an NVIDIA GPU with an NVIDIA Virtual GPU Software Driver version that supports this functionality. Refer to the relevant NVIDIA vGPU documentation for more details.
• You have a correctly configured NVIDIA vGPU assigned to a VM.

• All of the vGPU migration prerequisites that are documented by NVIDIA. Refer to the relevant NVIDIA vGPU documentation for more details.
• All of the general VM migration prerequisites. For details, see Migrating a virtual machine by using the command line

Limitations

• Certain NVIDIA GPU features can disable the migration. For more information, see the specific NVIDIA documentation for your graphics card.
• Some GPU workloads are not compatible with the downtime that happens during a migration. As a consequence, the GPU workloads might stop or crash. It is recommended to test if your workloads are compatible with the downtime before attempting a vGPU live migration.
• Currently, vGPU live migration fails if the vGPU driver version differs on the source and destination hosts.

• Currently, some general virtualization features cannot be used when live migrating a VM with an attached vGPU:

  • Calculating dirty memory page rate generation of the VM.

    Currently, live migration data and statistics provided by virsh domjobinfo and virsh domdirtyrate-calc commands are inaccurate when migrating a VM with an attached vGPU, because the calculations only count guest RAM without including vRAM from the vGPU.

  • Using a post-copy live migration.
  • Using a virtual I/O Memory Management Unit (vIOMMU) device in the VM.

Procedure

• For instructions on how to proceed with the live migration, see: Migrating a virtual machine by using the command line

  No additional parameters for the migration command are required for the attached vGPU device.

Prerequisites

• The VM intended for migration is shut down.
• Optional: A host system is available for hosting the storage that is not the source or destination host, but both the source and the destination host can reach it through the network. This is the optimal solution for shared storage.
• Make sure that NFS file locking is not used as it is not supported in KVM.
• The NFS protocol is installed and enabled on the source and destination hosts.

• The virt_use_nfs SELinux boolean is set to on.

  # setsebool virt_use_nfs 1

  Copy to Clipboard

Verification

• Start the VM on the source host and observe if it boots successfully.

Prerequisites

• Virtualization is installed and enabled on your system.
• You have administrator access to the source host and the destination host for the migration.

Next steps

• Sharing virtual machine disk images with other hosts
• Migrating a virtual machine by using the command line
• Live-migrating a virtual machine by using the web console

Prerequisites

• You have installed the RHEL 10 web console.

  For instructions, see Installing and enabling the web console.

• The web console VM plug-in is installed on your system.