Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 21. Creating nested virtual machines

You can use nested virtual machines (VMs) if you require a different host operating system than what your local host is running. This eliminates the need for additional physical hardware.

Warning

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

For detailed descriptions of the supported and unsupported environments, see Support limitations for nested virtualization.

21.1. What is nested virtualization?
Copy link

With nested virtualization, you can run virtual machines (VMs) within other VMs. A standard VM that runs on a physical host can also act as a second hypervisor and create its own VMs.

Nested virtualization terminology

Level 0 (L0)
A physical host, a bare-metal machine.
Level 1 (L1)
A standard VM, running on an L0 physical host, that can act as an additional virtual host.
Level 2 (L2)

A nested VM running on an L1 virtual host.

Important: The second level of virtualization severely limits the performance of an L2 VM. For this reason, nested virtualization is primarily intended for development and testing scenarios, such as:

  • Debugging hypervisors in a constrained environment
  • Testing larger virtual deployments on a limited amount of physical resources
Warning

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

For detailed descriptions of the supported and unsupported environments, see Support limitations for nested virtualization.

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

However, you can use a Windows virtual machine (VM) with the Windows Subsystem for Linux (WSL2) to create a virtual Linux environment inside the Windows VM. This use case is fully supported on RHEL 10 under specific conditions.

To learn more about the relevant terminology for nested virtualization, see What is nested virtualization?

Supported environments

To create a supported deployment of nested virtualization, create an L1 Windows VM on a RHEL 9 or RHEL 10 L0 host and use WSL2 to create a virtual Linux environment inside the L1 Windows VM. Currently, this is the only supported nested environment.

Important

The L0 host must be an Intel or AMD system. Other architectures, such as ARM or IBM Z, are currently not supported.

You must use only the following operating system versions:

Expand
On the L0 host:On the L1 VMs:

RHEL 10.0 and later

Windows Server 2019 and later with WSL2

 

Windows 10 and later with WSL2

See Microsoft documentation for instructions on installing WSL2 and choosing supported Linux distributions.

To create a supported nested environment, use one of the following procedures:

Technology Preview environments

These nested environments are available only as a Technology Preview and are not supported.

Important

The L0 host must be an Intel, AMD, or IBM Z system. Nested virtualization currently does not work on other architectures, such as ARM.

You must use only the following operating system versions for the deployment to work:

Expand
On the L0 host:On the L1 VMs:On the L2 VMs:

RHEL 10.0 and later

RHEL 9.6 and later

RHEL 9.6 and later

 

RHEL 10.0 and later

RHEL 10.0 and later

 

Windows Server 2016 and later with Hyper-V

Windows Server 2019 and later

 

Windows 10 and later with Hyper-V

 
Note

Creating RHEL L1 VMs is not tested when used in other Red Hat virtualization offerings. These include:

  • Red Hat Virtualization
  • Red Hat OpenStack Platform
  • OpenShift Virtualization

To create a Technology Preview nested environment, use one of the following procedures:

Hypervisor limitations

  • Currently, Red Hat tests nesting only on RHEL-KVM. When RHEL is used as the L0 hypervisor, you can use RHEL or Windows as the L1 hypervisor.
  • When using an L1 RHEL VM on a non-KVM L0 hypervisor, such as VMware ESXi or Amazon Web Services (AWS), creating L2 VMs in the RHEL guest operating system has not been tested and might not work.

Feature limitations

  • Use of L2 VMs as hypervisors and creating L3 guests has not been properly tested and is not expected to work.
  • Migrating VMs currently does not work on AMD systems if nested virtualization has been enabled on the L0 host.

  • On an IBM Z system, huge-page backing storage and nested virtualization cannot be used at the same time. 

    # modprobe kvm hpage=1 nested=1
modprobe: ERROR: could not insert 'kvm': Invalid argument
# dmesg |tail -1
[90226.508366] kvm-s390: A KVM host that supports nesting cannot back its KVM guests with huge pages
    Copy to Clipboard Toggle word wrap
  • Some features available on the L0 host might be unavailable on the L1 hypervisor.

21.3. Creating a nested virtual machine on Intel
Copy link

Follow the steps below to enable and configure nested virtualization on an Intel host.

Warning

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

For detailed descriptions of the supported and unsupported environments, see Support limitations for nested virtualization.

Prerequisites

  • An L0 RHEL 10 host running an L1 virtual machine (VM).
  • The hypervisor CPU must support nested virtualization. To verify, use the cat /proc/cpuinfo command on the L0 hypervisor. If the output of the command includes the vmx and ept flags, creating L2 VMs is possible. This is generally the case on Intel Xeon v3 cores and later.

  • Ensure that nested virtualization is enabled on the L0 host: 

    # cat /sys/module/kvm_intel/parameters/nested
    Copy to Clipboard Toggle word wrap
    • If the command returns 1 or Y, the feature is enabled. Skip the remaining prerequisite steps, and continue with the Procedure section.

    • If the command returns 0 or N but your system supports nested virtualization, use the following steps to enable the feature.

      1. Unload the kvm_intel module: 

        # modprobe -r kvm_intel
        Copy to Clipboard Toggle word wrap

      2. Activate the nesting feature: 

        # modprobe kvm_intel nested=1
        Copy to Clipboard Toggle word wrap

      3. The nesting feature is now enabled, but only until the next reboot of the L0 host. To enable it permanently, add the following line to the /etc/modprobe.d/kvm.conf file: 

        options kvm_intel nested=1
        Copy to Clipboard Toggle word wrap

Procedure

  1. Configure your L1 VM for nested virtualization.

    1. Open the XML configuration of the VM. The following example opens the configuration of the Intel-L1 VM: 

      # virsh edit Intel-L1
      Copy to Clipboard Toggle word wrap

    2. Configure the VM to use host-passthrough CPU mode by editing the <cpu> element: 

      <cpu mode='host-passthrough'/>
      Copy to Clipboard Toggle word wrap

      If you require the VM to use a specific CPU model, configure the VM to use custom CPU mode. Inside the <cpu> element, add a <feature policy='require' name='vmx'/> element and a <model> element with the CPU model specified inside. For example: 

      <cpu mode ='custom' match ='exact' check='partial'>
  <model fallback='allow'>Haswell-noTSX</model>
  <feature policy='require' name='vmx'/>
  ...
</cpu>
      Copy to Clipboard Toggle word wrap
  2. Create an L2 VM within the L1 VM. To do this, follow the same procedure as when creating the L1 VM.

21.4. Creating a nested virtual machine on AMD
Copy link

Follow the steps below to enable and configure nested virtualization on an AMD host.

Warning

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

For detailed descriptions of the supported and unsupported environments, see Support limitations for nested virtualization.

Prerequisites

  • An L0 RHEL 10 host running an L1 virtual machine (VM).
  • The hypervisor CPU must support nested virtualization. To verify, use the cat /proc/cpuinfo command on the L0 hypervisor. If the output of the command includes the svm and npt flags, creating L2 VMs is possible. This is generally the case on AMD EPYC cores and later.

  • Ensure that nested virtualization is enabled on the L0 host: 

    # cat /sys/module/kvm_amd/parameters/nested
    Copy to Clipboard Toggle word wrap
    • If the command returns 1 or Y, the feature is enabled. Skip the remaining prerequisite steps, and continue with the Procedure section.

    • If the command returns 0 or N, use the following steps to enable the feature.

      1. Stop all running VMs on the L0 host.

      2. Unload the kvm_amd module: 

        # modprobe -r kvm_amd
        Copy to Clipboard Toggle word wrap

      3. Activate the nesting feature: 

        # modprobe kvm_amd nested=1
        Copy to Clipboard Toggle word wrap

      4. The nesting feature is now enabled, but only until the next reboot of the L0 host. To enable it permanently, add the following to the /etc/modprobe.d/kvm.conf file: 

        options kvm_amd nested=1
        Copy to Clipboard Toggle word wrap

Procedure

  1. Configure your L1 VM for nested virtualization.

    1. Open the XML configuration of the VM. The following example opens the configuration of the AMD-L1 VM: 

      # virsh edit AMD-L1
      Copy to Clipboard Toggle word wrap

    2. Configure the VM to use host-passthrough CPU mode by editing the <cpu> element: 

      <cpu mode='host-passthrough'/>
      Copy to Clipboard Toggle word wrap

      If you require the VM to use a specific CPU model, configure the VM to use custom CPU mode. Inside the <cpu> element, add a <feature policy='require' name='svm'/> element and a <model> element with the CPU model specified inside. For example: 

      <cpu mode="custom" match="exact" check="none">
  <model fallback="allow">EPYC-IBPB</model>
  <feature policy="require" name="svm"/>
  ...
</cpu>
      Copy to Clipboard Toggle word wrap
  2. Create an L2 VM within the L1 VM. To do this, follow the same procedure as when creating the L1 VM.

21.5. Creating a nested virtual machine on IBM Z
Copy link

Follow the steps below to enable and configure nested virtualization on an IBM Z host.

Note

IBM Z does not really provide a bare-metal L0 host. Instead, user systems are set up on a logical partition (LPAR), which is already a virtualized system, so it is often referred to as L1. However, for better alignment with other architectures in this guide, the following steps refer to IBM Z as if it provides an L0 host.

To learn more about nested virtualization, see: What is nested virtualization?

Warning

In most environments, nested virtualization is only available as a Technology Preview in RHEL 10.

For detailed descriptions of the supported and unsupported environments, see Support limitations for nested virtualization.

Prerequisites

  • An L0 RHEL 10 host running an L1 virtual machine (VM).
  • The hypervisor CPU must support nested virtualization. To verify this is the case, use the cat /proc/cpuinfo command on the L0 hypervisor. If the output of the command includes the sie flag, creating L2 VMs is possible.

  • Ensure that nested virtualization is enabled on the L0 host: 

    # cat /sys/module/kvm/parameters/nested
    Copy to Clipboard Toggle word wrap
    • If the command returns 1 or Y, the feature is enabled. Skip the remaining prerequisite steps, and continue with the Procedure section.

    • If the command returns 0 or N, use the following steps to enable the feature.

      1. Stop all running VMs on the L0 host.

      2. Unload the kvm module: 

        # modprobe -r kvm
        Copy to Clipboard Toggle word wrap

      3. Activate the nesting feature: 

        # modprobe kvm nested=1
        Copy to Clipboard Toggle word wrap

      4. The nesting feature is now enabled, but only until the next reboot of the L0 host. To enable it permanently, add the following line to the /etc/modprobe.d/kvm.conf file: 

        options kvm nested=1
        Copy to Clipboard Toggle word wrap

Procedure

  • Create an L2 VM within the L1 VM. To do this, follow the same procedure as when creating the L1 VM.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat