Red Hat Summit Red Hat SummitSuporteConsoleDesenvolvedoresComeçar um testeContato

Este conteúdo não está disponível no idioma selecionado.

Chapter 24. Feature support and limitations in RHEL 9 virtualization

This document provides information about feature support and restrictions in Red Hat Enterprise Linux 9 (RHEL 9) virtualization.

24.1. How RHEL virtualization support works

A set of support limitations applies to virtualization in Red Hat Enterprise Linux 9 (RHEL 9). This means that when you use certain features or exceed a certain amount of allocated resources when using virtual machines in RHEL 9, Red Hat will not support these guests unless you have a specific subscription plan.

Features listed in Recommended features in RHEL 9 virtualization have been tested and certified by Red Hat to work with the KVM hypervisor on a RHEL 9 system. Therefore, they are fully supported and recommended for use in virtualization in RHEL 9.

Features listed in Unsupported features in RHEL 9 virtualization may work, but are not supported and not intended for use in RHEL 9. Therefore, Red Hat strongly recommends not using these features in RHEL 9 with KVM.

Resource allocation limits in RHEL 9 virtualization lists the maximum amount of specific resources supported on a KVM guest in RHEL 9. Guests that exceed these limits are not supported by Red Hat.

In addition, unless stated otherwise, all features and solutions used by the documentation for RHEL 9 virtualization are supported. However, some of these have not been completely tested and therefore may not be fully optimized.

Important

Many of these limitations do not apply to other virtualization solutions provided by Red Hat, such as OpenShift Virtualization or Red Hat OpenStack Platform (RHOSP).

24.3. Unsupported features in RHEL 9 virtualization

The following features are not supported by the KVM hypervisor included with Red Hat Enterprise Linux 9 (RHEL 9):

Important

Many of these limitations may not apply to other virtualization solutions provided by Red Hat, such as OpenShift Virtualization or Red Hat OpenStack Platform (RHOSP).

Features supported by other virtualization solutions are described as such in the following paragraphs.

Host system architectures

RHEL 9 with KVM is not supported on any host architectures that are not listed in Recommended features in RHEL 9 virtualization.

Guest operating systems

KVM virtual machines (VMs) that use the following guest operating systems (OSs) are not supported on a RHEL 9 host:

  • Windows 8.1 and earlier
  • Windows Server 2012 R2 and earlier
  • macOS
  • Solaris for x86 systems
  • Any operating system released before 2009

For a list of guest OSs supported on RHEL hosts and other virtualization solutions, see Certified Guest Operating Systems in Red Hat OpenStack Platform, Red Hat Virtualization, OpenShift Virtualization and Red Hat Enterprise Linux with KVM.

Creating VMs in containers

Red Hat does not support creating KVM virtual machines in any type of container that includes the elements of the RHEL 9 hypervisor (such as the QEMU emulator or the libvirt package).

To create VMs in containers, Red Hat recommends using the OpenShift Virtualization offering.

Specific virsh commands and options

Not every parameter that you can use with the virsh utility has been tested and certified as production-ready by Red Hat. Therefore, any virsh commands and options that are not explicitly recommended by Red Hat documentation may not work correctly, and Red Hat recommends not using them in your production environment.

Notably, unsupported virsh commands include the following:

  • virsh iface-* commands, such as virsh iface-start and virsh iface-destroy
  • virsh blkdeviotune
  • virsh snapshot-* commands, such as virsh snapshot-create and virsh snapshot-revert

The QEMU command line

QEMU is an essential component of the virtualization architecture in RHEL 9, but it is difficult to manage manually, and improper QEMU configurations might cause security vulnerabilities. Therefore, using qemu-* command-line utilities, such as, qemu-kvm is not supported by Red Hat. Instead, use libvirt utilities, such as virt-install, virt-xml, and supported virsh commands, as these orchestrate QEMU according to the best practices. However, the qemu-img utility is supported for management of virtual disk images.

vCPU hot unplug

Removing a virtual CPU (vCPU) from a running VM, also referred to as a vCPU hot unplug, is not supported in RHEL 9.

Memory hot unplug

Removing a memory device attached to a running VM, also referred to as a memory hot unplug, is unsupported in RHEL 9.

QEMU-side I/O throttling

Using the virsh blkdeviotune utility to configure maximum input and output levels for operations on virtual disk, also known as QEMU-side I/O throttling, is not supported in RHEL 9.

To set up I/O throttling in RHEL 9, use virsh blkiotune. This is also known as libvirt-side I/O throttling. For instructions, see Disk I/O throttling in virtual machines.

Other solutions:

  • QEMU-side I/O throttling is also supported in RHOSP. For more information, see Red Hat Knowledgebase solutions Setting Resource Limitation on Disk and the Use Quality-of-Service Specifications section in the RHOSP Storage Guide.
  • In addition, OpenShift Virtualizaton supports QEMU-side I/O throttling as well.

Storage live migration

Migrating a disk image of a running VM between hosts is not supported in RHEL 9.

Other solutions:

  • Storage live migration is supported in RHOSP, but with some limitations. For details, see Migrate a Volume.

Internal snapshots

Creating and using internal snapshots for VMs is deprecated in RHEL 9, and highly discouraged for use in production environment. Instead, use external snapshots. For details, see Support limitations for virtual machine snapshots.

Other solutions:

vHost Data Path Acceleration

On RHEL 9 hosts, it is possible to configure vHost Data Path Acceleration (vDPA) for virtio devices, but Red Hat currently does not support this feature, and strongly discourages its use in production environments.

vhost-user

RHEL 9 does not support the implementation of a user-space vHost interface.

Other solutions:

  • vhost-user is supported in RHOSP, but only for virtio-net interfaces. For more information, see the Red Hat Knowledgebase solution virtio-net implementation and vhost user ports.
  • OpenShift Virtualization supports vhost-user as well.

S3 and S4 system power states

Suspending a VM to the Suspend to RAM (S3) or Suspend to disk (S4) system power states is not supported. Note that these features are disabled by default, and enabling them will make your VM not supportable by Red Hat.

Note that the S3 and S4 states are also currently not supported in any other virtualization solution provided by Red Hat.

S3-PR on a multipathed vDisk

SCSI3 persistent reservation (S3-PR) on a multipathed vDisk is not supported in RHEL 9. As a consequence, Windows Cluster is not supported in RHEL 9.

virtio-crypto

Using the virtio-crypto device in RHEL 9 is not supported and RHEL strongly discourages its use.

Note that virtio-crypto devices are also not supported in any other virtualization solution provided by Red Hat.

virtio-multitouch-device, virtio-multitouch-pci

Using the virtio-multitouch-device and virtio-multitouch-pci devices in RHEL 9 is not supported and RHEL strongly discourages their use.

Incremental live backup

Configuring a VM backup that only saves VM changes since the last backup, also known as incremental live backup, is not supported in RHEL 9, and Red Hat highly discourages its use.

net_failover

Using the net_failover driver to set up an automated network device failover mechanism is not supported in RHEL 9.

Note that net_failover is also currently not supported in any other virtualization solution provided by Red Hat.

TCG

QEMU and libvirt include a dynamic translation mode using the QEMU Tiny Code Generator (TCG). This mode does not require hardware virtualization support. However, TCG is not supported by Red Hat.

TCG-based guests can be recognized by examining its XML configuration, for example using the virsh dumpxml command.

  • The configuration file of a TCG guest contains the following line: 

    <domain type='qemu'>

  • The configuration file of a KVM guest contains the following line: 

    <domain type='kvm'>

SR-IOV InfiniBand networking devices

Attaching InfiniBand networking devices to VMs using Single-root I/O virtualization (SR-IOV) is not supported.

SGIO

Attaching SCSI devices to VMs by using SCSI generic I/O (SGIO) is not supported on RHEL 9. To detect whether your VM has an attached SGIO device, check the VM configuration for the following lines: 

<disk type="block" device="lun">
<hostdev mode='subsystem' type='scsi'>

Additional resources

24.4. Resource allocation limits in RHEL 9 virtualization

The following limits apply to virtualized resources that can be allocated to a single KVM virtual machine (VM) on a Red Hat Enterprise Linux 9 (RHEL 9) host.

Important

Many of these limitations do not apply to other virtualization solutions provided by Red Hat, such as OpenShift Virtualization or Red Hat OpenStack Platform (RHOSP).

Maximum vCPUs per VM

For the maximum amount of vCPUs and memory that is supported on a single VM running on a RHEL 9 host, see: Virtualization limits for Red Hat Enterprise Linux with KVM

PCI devices per VM

RHEL 9 supports 32 PCI device slots per VM bus, and 8 PCI functions per device slot. This gives a theoretical maximum of 256 PCI functions per bus when multi-function capabilities are enabled in the VM, and no PCI bridges are used.

Each PCI bridge adds a new bus, potentially enabling another 256 device addresses. However, some buses do not make all 256 device addresses available for the user; for example, the root bus has several built-in devices occupying slots.

Virtualized IDE devices

KVM is limited to a maximum of 4 virtualized IDE devices per VM.

24.5. Supported disk image formats

To run a virtual machine (VM) on RHEL, you must use a disk image with a supported format. You can also convert certain unsupported disk images to a supported format.

Supported disk image formats for VMs

You can use disk images that use the following formats to run VMs in RHEL:

  • qcow2 - Provides certain additional features, such as compression.
  • raw - Might provide better performance.
  • luks - Disk images encrypted by using the Linux Unified Key Setup (LUKS) specification.

Supported disk image formats for conversion

  • If required, you can convert your disk images between the raw and qcow2 formats by using the qemu-img convert command.
  • If you require converting a vmdk disk image to a raw or qcow2 format, convert the VM that uses the disk to KVM by using the virt-v2v utility.

  • To convert other disk image formats to raw or qcow2, you can use the qemu-img convert command. For a list of formats that work with this command, see the QEMU documentation.

    Note that in most cases, converting the disk image format of a non-KVM virtual machine to qcow2 or raw is not sufficient for the VM to correctly run on RHEL KVM. In addition to converting the disk image, corresponding drivers must be installed and configured in the guest operating system of the VM. For supported hypervisor conversion, use the virt-v2v utility.

Additional resources

24.6. How virtualization on IBM Z differs from AMD64 and Intel 64

KVM virtualization in RHEL 9 on IBM Z systems differs from KVM on AMD64 and Intel 64 systems in the following:

PCI and USB devices

Virtual PCI and USB devices are not supported on IBM Z. This also means that virtio-*-pci devices are unsupported, and virtio-*-ccw devices should be used instead. For example, use virtio-net-ccw instead of virtio-net-pci.

Note that direct attachment of PCI devices, also known as PCI passthrough, is supported.

Supported guest operating system
Red Hat only supports VMs hosted on IBM Z if they use RHEL 7, 8, or 9 as their guest operating system.
Device boot order

IBM Z does not support the <boot dev='device'> XML configuration element. To define device boot order, use the <boot order='number'> element in the <devices> section of the XML.

Note

Using <boot order='number'> for boot order management is recommended on all host architectures.

In addition, you can select the required boot entry by using the architecture-specific loadparm attribute in the <boot> element. For example, the following determines that the disk should be used first in the boot sequence and if a Linux distribution is available on that disk, it will select the second boot entry: 

<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2'/>
  <source file='/path/to/qcow2'/>
  <target dev='vda' bus='virtio'/>
  <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
  <boot order='1' loadparm='2'/>
</disk>
Memory hot plug
Adding memory to a running VM is not possible on IBM Z. Note that removing memory from a running VM (memory hot unplug) is also not possible on IBM Z, as well as on AMD64 and Intel 64.
NUMA topology
Non-Uniform Memory Access (NUMA) topology for CPUs is not supported by libvirt on IBM Z. Therefore, tuning vCPU performance by using NUMA is not possible on these systems.
GPU devices
Assigning GPU devices is not supported on IBM Z systems.
vfio-ap
VMs on an IBM Z host can use the vfio-ap cryptographic device passthrough, which is not supported on any other architecture.
vfio-ccw
VMs on an IBM Z host can use the vfio-ccw disk device passthrough, which is not supported on any other architecture.
SMBIOS
SMBIOS configuration is not available on IBM Z.
Watchdog devices

If using watchdog devices in your VM on an IBM Z host, use the diag288 model. For example: 

<devices>
  <watchdog model='diag288' action='poweroff'/>
</devices>
kvm-clock
The kvm-clock service is specific to AMD64 and Intel 64 systems, and does not have to be configured for VM time management on IBM Z.
v2v and p2v
The virt-v2v and virt-p2v utilities are supported only on the AMD64 and Intel 64 architecture, and are not provided on IBM Z.
Migrations

To successfully migrate to a later host model (for example from IBM z14 to z15), or to update the hypervisor, use the host-model CPU mode. The host-passthrough and maximum CPU modes are not recommended, as they are generally not migration-safe.

If you want to specify an explicit CPU model in the custom CPU mode, follow these guidelines:

  • Do not use CPU models that end with -base.
  • Do not use the qemu, max or host CPU model.

To successfully migrate to an older host model (such as from z15 to z14), or to an earlier version of QEMU, KVM, or the RHEL kernel, use the CPU type of the oldest available host model without -base at the end.

PXE installation and booting

When using PXE to run a VM on IBM Z, a specific configuration is required for the pxelinux.cfg/default file. For example: 

# pxelinux
default linux
label linux
kernel kernel.img
initrd initrd.img
append ip=dhcp inst.repo=example.com/redhat/BaseOS/s390x/os/
Secure Execution
You can boot a VM with a prepared secure guest image by defining <launchSecurity type="s390-pv"/> in the XML configuration of the VM. This encrypts the VM’s memory to protect it from unwanted access by the hypervisor.

Note that the following features are not supported when running a VM in secure execution mode:

  • Device passthrough by using vfio
  • Obtaining memory information by using virsh domstats and virsh memstat
  • The memballoon and virtio-rng virtual devices
  • Memory backing by using huge pages
  • Live and non-live VM migrations
  • Saving and restoring VMs
  • VM snapshots, including memory snapshots (using the --memspec option)
  • Full memory dumps. Instead, specify the --memory-only option for the virsh dump command.
  • 248 or more vCPUs. The vCPU limit for secure guests is 247.

Additional resources

24.7. How virtualization on ARM 64 differs from AMD64 and Intel 64

KVM virtualization in RHEL 9 on ARM 64 systems (also known as AArch64) is different from KVM on AMD64 and Intel 64 systems in several aspects. These include, but are not limited to, the following:

Guest operating systems
The only guest operating system currently supported on ARM 64 virtual machines (VMs) is RHEL 9.
vCPU hot plug and hot unplug
Attaching a virtual CPU (vCPU) to a running VM, also referred to as a vCPU hot plug, is currently not supported on ARM 64 hosts. In addition, like on AMD64 and Intel 64 hosts, removing a vCPU from a running VM (vCPU hot unplug), is not supported on ARM 64.
SecureBoot
The SecureBoot feature is not available on ARM 64 systems.
Migration
Migrating VMs between ARM 64 hosts is currently not supported.
Saving and restoring VMs
Saving and restoring a VM is currently unsupported on an ARM 64 host.
Memory page sizes

ARM 64 currently supports running VMs with 64 KB or 4 KB memory page sizes, however both the host and the guest must use the same memory page size. Configurations where host and guest have different memory page sizes are not supported.

By default, RHEL 9 uses a 4 KB memory page size. If you want to run a VM with a 64 KB memory page size, your host must be using a kernel with 64 KB memory page size. When creating the VM, you must install it with the kernel-64k package, for example by including the following parameter in the kickstart file: 

%packages
-kernel
kernel-64k
%end
Huge pages

ARM 64 hosts with 64 KB memory page size support huge memory pages with the following sizes:

  • 2 MB
  • 512 MB

  • 16 GB

    When you use transparent huge pages (THP) on an ARM 64 host with 64 KB memory page size, it supports only 512 MB huge pages.

    ARM 64 hosts with 4 KB memory page size support huge memory pages with the following sizes:

  • 64 KB
  • 2 MB
  • 32 MB

  • 1024 MB

    When you use transparent huge pages (THP) on an ARM 64 host with 4 KB memory page size, it supports only 2 MB huge pages.

SVE

The ARM 64 architecture provides the Scalable Vector Expansion (SVE) feature. If the host supports the feature, using SVE in your VMs improves the speed of vector mathematics computation and string operations in these VMs.

The base-line level of SVE is enabled by default on host CPUs that support it. However, Red Hat recommends configuring each vector length explicitly. This ensures that the VM can only be launched on compatible hosts. To do so:

  1. Verify that your CPU has the SVE feature: 

    # grep -m 1 Features /proc/cpuinfo | grep -w sve

Features: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm fcma dcpop sve

    If the output of this command includes sve or if its exit code is 0, your CPU supports SVE.

  2. Open the XML configuration of the VM you want to modify: 

    # virsh edit vm-name

  3. Edit the <cpu> element similarly to the following: 

    <cpu mode='host-passthrough' check='none'>
<feature policy='require' name='sve'/>
<feature policy='require' name='sve128'/>
<feature policy='require' name='sve256'/>
<feature policy='disable' name='sve384'/>
<feature policy='require' name='sve512'/>
</cpu>

    This example explicitly enables SVE vector lengths 128, 256, and 512, and explicitly disables vector length 384.

CPU models
VMs on ARM 64 currently only support the host-passthrough CPU model.
PXE

Booting in the Preboot Execution Environment (PXE) is functional but not supported, Red Hat strongly discourages using it in production environments.

If you require PXE booting, it is only possible with the virtio-net-pci network interface controller (NIC).

EDK2

ARM 64 guests use UEFI firmware included in the edk2-aarch64 package, which provides a similar interface as OVMF UEFI on AMD64 and Intel 64, and implements a similar set of features.

Specifically, edk2-aarch64 provides a built-in UEFI shell, but does not support the following functionality:

  • SecureBoot
  • Management Mode
kvm-clock
The kvm-clock service does not have to be configured for time management in VMs on ARM 64.
Peripheral devices

ARM 64 systems support a partly different set of peripheral devices than AMD64 and Intel 64 devices.

  • Only PCIe topologies are supported.
  • ARM 64 systems support virtio devices by using the virtio-*-pci drivers. In addition, the virtio-iommu and virtio-input devices are unsupported.
  • The virtio-gpu driver is only supported for graphical installs.
  • ARM 64 systems support usb-mouse and usb-tablet devices for graphical installs only. Other USB devices, USB passthrough, or USB redirect are not supported.
  • Device assignment that uses Virtual Function I/O (VFIO) is supported only for NICs (physical and virtual functions).
Emulated devices

The following devices are not supported on ARM 64:

  • Emulated sound devices, such as ICH9, ICH6 or AC97.
  • Emulated graphics cards, such as VGA cards.
  • Emulated network devices, such as rtl8139.
GPU devices
Assigning GPU devices is currently not supported on ARM 64 systems.
Nested virtualization
Creating nested VMs is currently not possible on ARM 64 hosts.
v2v and p2v
The virt-v2v and virt-p2v utilities are only supported on the AMD64 and Intel 64 architecture and are, therefore, not provided on ARM 64.

24.8. An overview of virtualization features support in RHEL 9

The following tables provide comparative information about the support state of selected virtualization features in RHEL 9 across the available system architectures.

Table 24.1. General support
Intel 64 and AMD64IBM ZARM 64

Supported

Supported

Supported

Table 24.2. Device hot plug and hot unplug
 Intel 64 and AMD64IBM ZARM 64

CPU hot plug

Supported

Supported

UNSUPPORTED

CPU hot unplug

UNSUPPORTED

UNSUPPORTED

UNSUPPORTED

Memory hot plug

Supported

UNSUPPORTED

Supported

Memory hot unplug

UNSUPPORTED

UNSUPPORTED

UNSUPPORTED

Peripheral device hot plug

Supported

Supported [a]

Supported

Peripheral device hot unplug

Supported

Supported [b]

Supported

[a] Requires using virtio-*-ccw devices instead of virtio-*-pci
[b] Requires using virtio-*-ccw devices instead of virtio-*-pci
Table 24.3. Other selected features
 Intel 64 and AMD64IBM ZARM 64

NUMA tuning

Supported

UNSUPPORTED

Supported

SR-IOV devices

Supported

UNSUPPORTED

Supported

virt-v2v and p2v

Supported

UNSUPPORTED

UNAVAILABLE

Note that some of the unsupported features are supported on other Red Hat products, such as Red Hat Virtualization and Red Hat OpenStack platform. For more information, see Unsupported features in RHEL 9 virtualization.

Additional sources

Red Hat logoGithubRedditYoutubeTwitter

Aprender

Experimente, compre e venda

Comunidades

Sobre a documentação da Red Hat

Ajudamos os usuários da Red Hat a inovar e atingir seus objetivos com nossos produtos e serviços com conteúdo em que podem confiar. Explore nossas atualizações recentes.

Tornando o open source mais inclusivo

A Red Hat está comprometida em substituir a linguagem problemática em nosso código, documentação e propriedades da web. Para mais detalhes veja o Blog da Red Hat.

Sobre a Red Hat

Fornecemos soluções robustas que facilitam o trabalho das empresas em plataformas e ambientes, desde o data center principal até a borda da rede.

© 2024 Red Hat, Inc.