Red Hat Summit1.2. Supported limits
Refer to tested object maximums when planning your environment for your specific use case, and consider all factors that can impact cluster scaling. For information options that impact performance, see the "OpenShift Virtualization - Tuning & Scaling Guide" in the Red Hat Knowledgebase.
1.2.1. Tested maximums for OpenShift Virtualization
Maximums are based on a single cluster of the largest possible size for a OpenShift Virtualization 4.x environment. Approaching the maximum values can reduce performance and increase latency. Consider using multiple smaller clusters if possible to improve performance.
1.2.1.1. Virtual machine maximums
Maximums apply to virtual machines (VMs) running on OpenShift Virtualization and are subject to the limits specified in "Virtualization limits for Red Hat Enterprise Linux with KVM".
| Objective (per VM) | Tested limit | Theoretical limit |
|---|---|---|
| Virtual CPUs | 255 vCPUs | 255 vCPUs |
| Memory | 6 TB | 16 TB |
| Single disk size | 100 TB | 100 TB |
| Hot-pluggable disks | 255 disks | N/A |
Each VM must have at least 512 MB of memory. The fstype in the guest operating system (OS) must support the maximum limits. Do not use preallocation in data volumes that are larger than 99 TB.
1.2.1.2. Host maximums
The following maximums apply to the OpenShift Container Platform hosts used for OpenShift Virtualization.
| Objective (per host) | Tested limit | Theoretical limit |
|---|---|---|
| Logical CPU cores or threads | Same as Red Hat Enterprise Linux (RHEL) | N/A |
| RAM | Same as RHEL | N/A |
| Simultaneous live migrations | Defaults to 2 outbound migrations per node, and 5 concurrent migrations per cluster | Depends on NIC bandwidth |
| Live migration bandwidth | No default limit | Depends on NIC bandwidth |
1.2.1.3. Cluster maximums
The following maximums apply to objects defined in OpenShift Virtualization.
| Objective (per cluster) | Tested limit | Theoretical limit |
|---|---|---|
| Number of attached PVs per node | N/A | CSI storage provider dependent |
| Maximum PV size | N/A | CSI storage provider dependent |
| Hosts | 500 hosts (100 or fewer recommended) [1] | Same as OpenShift Container Platform |
| Defined VMs | 10,000 VMs [2] | Same as OpenShift Container Platform |
If you use more than 100 nodes, consider using Red Hat Advanced Cluster Management (RHACM) to manage multiple clusters instead of scaling out a single control plane. Larger clusters add complexity, require longer updates, and depending on node size and total object density, they can increase control plane stress.
Using multiple clusters can be beneficial in areas like per-cluster isolation and high availability.
The maximum number of VMs per node depends on the host hardware and resource capacity. It is also limited by the following parameters:
-
Settings that limit the number of pods that can be scheduled to a node. For example:
maxPods. -
The default number of KVM devices. For example:
devices.kubevirt.io/kvm: 1k.
-
Settings that limit the number of pods that can be scheduled to a node. For example:
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}