Este contenido no está disponible en el idioma seleccionado.
OpenShift Virtualization
OpenShift Virtualization installation, usage, and release notes
Abstract
Chapter 1. About OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
Learn about OpenShift Virtualization’s capabilities and support scope.
1.1. What you can do with OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization is an add-on to OpenShift Container Platform that allows you to run and manage virtual machine workloads alongside container workloads.
OpenShift Virtualization adds new objects into your OpenShift Container Platform cluster via Kubernetes custom resources to enable virtualization tasks. These tasks include:
- Creating and managing Linux and Windows virtual machines
- Connecting to virtual machines through a variety of consoles and CLI tools
- Importing and cloning existing virtual machines
- Managing network interface controllers and storage disks attached to virtual machines
- Live migrating virtual machines between nodes
An enhanced web console provides a graphical portal to manage these virtualized resources alongside the OpenShift Container Platform cluster containers and infrastructure.
OpenShift Virtualization is tested with OpenShift Container Storage (OCS) and designed to use with OCS features for the best experience.
You can use OpenShift Virtualization with the OVN-Kubernetes, OpenShift SDN, or one of the other certified default Container Network Interface (CNI) network providers listed in Certified OpenShift CNI Plugins.
1.1.1. OpenShift Virtualization supported cluster version Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization 4.8 is supported for use on OpenShift Container Platform 4.8 clusters. To use the latest z-stream release of OpenShift Virtualization, you must first upgrade to the latest version of OpenShift Container Platform.
Chapter 2. Start here with OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
Use the following tables to find content to help you learn about and use OpenShift Virtualization.
2.1. Cluster administrator Copiar enlaceEnlace copiado en el portapapeles!
| Learn | Plan | Deploy | Additional resources |
|---|---|---|---|
| Installing OpenShift Virtualization using the OpenShift Virtualization console or CLI | |||
2.2. Virtualization administrator Copiar enlaceEnlace copiado en el portapapeles!
| Learn | Deploy | Manage | Use |
|---|---|---|---|
| Connecting virtual machines to the default pod network for virtual machines and external networks | Importing virtual machines with the Migration Toolkit for containers | ||
2.3. Virtual machine administrator / developer Copiar enlaceEnlace copiado en el portapapeles!
| Learn | Use | Manage | Additional resources |
|---|---|---|---|
| Pass configuration data to virtual machines using secrets, configuration maps, and service accounts |
Chapter 3. OpenShift Virtualization release notes Copiar enlaceEnlace copiado en el portapapeles!
3.1. About Red Hat OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
Red Hat OpenShift Virtualization enables you to bring traditional virtual machines (VMs) into OpenShift Container Platform where they run alongside containers, and are managed as native Kubernetes objects.
OpenShift Virtualization is represented by the
logo.
You can use OpenShift Virtualization with either the OVN-Kubernetes or the OpenShiftSDN default Container Network Interface (CNI) network provider.
Learn more about what you can do with OpenShift Virtualization.
3.1.1. OpenShift Virtualization supported cluster version Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization 4.8 is supported for use on OpenShift Container Platform 4.8 clusters. To use the latest z-stream release of OpenShift Virtualization, you must first upgrade to the latest version of OpenShift Container Platform.
3.1.2. Supported guest operating systems Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization guests can use the following operating systems:
- Red Hat Enterprise Linux 6, 7, and 8.
- Microsoft Windows Server 2012 R2, 2016, and 2019.
- Microsoft Windows 10.
Other operating system templates shipped with OpenShift Virtualization are not supported.
3.2. Making open source more inclusive Copiar enlaceEnlace copiado en el portapapeles!
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
3.3. New and changed features Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization is certified in Microsoft’s Windows Server Virtualization Validation Program (SVVP) to run Windows Server workloads.
The SVVP Certification applies to:
- Red Hat Enterprise Linux CoreOS workers. In the Microsoft SVVP Catalog, they are named Red Hat OpenShift Container Platform 4 on RHEL CoreOS.
- Intel and AMD CPUs.
- The Containerized Data Importer (CDI) now uses the OpenShift Container Platform cluster-wide proxy configuration.
- OpenShift Virtualization now supports third-party Container Network Interface (CNI) plug-ins that are certified by Red Hat for use with OpenShift Container Platform.
OpenShift Virtualization now provides metrics for monitoring how infrastructure resources are consumed in the cluster. You can use the OpenShift Container Platform monitoring dashboard to query metrics for the following resources:
- vCPU
- Network
- Storage
- Guest memory swapping
- OpenShift Virtualization now provides a unified API to configure certificate rotation.
- If a Windows virtual machine is created from a template or has predefined Hyper-V capabilities, it can now only be scheduled to Hyper-V capable nodes.
-
The option for the
--proxy-onlycommand allows you to manually connect to a virtual machine instance through a Virtual Network Client (VNC) connection by using any VNC viewer.virtctl vnc
3.3.1. Quick starts Copiar enlaceEnlace copiado en el portapapeles!
-
Quick start tours are available for several OpenShift Virtualization features. To view the tours, click the Help icon ? in the menu bar on the header of the OpenShift Virtualization console and then select Quick Starts. You can filter the available tours by entering the keyword in the Filter field.
virtualization
3.3.2. Networking Copiar enlaceEnlace copiado en el portapapeles!
- You can use the Kubernetes NMstate Operator to configure and manage IP addresses on your cluster nodes.
-
OpenShift Virtualization now supports live migration of virtual machines that are attached to an SR-IOV network interface if the feature gate is enabled in the
sriovLiveMigrationcustom resource (CR).HyperConverged
3.3.3. Storage Copiar enlaceEnlace copiado en el portapapeles!
- Cloning a data volume into a different namespace is now faster and more efficient when using storage that supports Container Storage Interface (CSI) snapshots. The Containerized Data Importer (CDI) uses CSI snapshots, when they are available, to improve performance when you create a virtual machine from a template.
-
When the or
fstrimcommands are run on a virtual disk, the discard requests are passed to the underlying storage device. If the storage provider supports the Pass Discard feature, the discard requests free up storage capacity.blkdiscard
-
You can now specify data volumes by using the storage API. The storage API, unlike the PVC API, allows the system to optimize ,
accessModes, and storage capacity when allocating storage.volumeMode
-
You can now clone virtual machine disks between different data volume modes if they have the content type . For example, you can clone a persistent volume (PV) with
kubevirtto a PV withvolumeMode: BlockvolumeMode: Filesystem.
- You can create a custom disk image as a boot source for any template that has a defined source by running a wizard in the OpenShift Virtualization console.
3.4. Deprecated and removed features Copiar enlaceEnlace copiado en el portapapeles!
3.4.1. Deprecated features Copiar enlaceEnlace copiado en el portapapeles!
Deprecated features are included in the current release and supported. However, they will be removed in a future release and are not recommended for new deployments.
- Importing a single virtual machine from Red Hat Virtualization (RHV) or VMware is deprecated in the current release and will be removed in OpenShift Virtualization 4.9. This feature is replaced by the Migration Toolkit for Virtualization.
3.5. Notable technical changes Copiar enlaceEnlace copiado en el portapapeles!
- OpenShift Virtualization now configures IPv6 addresses when running on clusters that have dual-stack networking enabled. You can create a service that uses IPv4, IPv6, or both IP address families, if dual-stack networking is enabled for the underlying OpenShift Container Platform cluster.
-
KubeMacPool is now enabled by default when you install OpenShift Virtualization. You can disable a MAC address pool for a namespace by adding the label to the namespace. Re-enable KubeMacPool for the namespace by removing the label.
mutatevirtualmachines.kubemacpool.io=ignore
The
custom resource (CR) is now the central point of configuration for OpenShift Virtualization. By editing theHyperConvergedCR, you can:HyperConverged
3.6. Technology Preview features Copiar enlaceEnlace copiado en el portapapeles!
Some features in this release are currently in Technology Preview. These experimental features are not intended for production use. Note the following scope of support on the Red Hat Customer Portal for these features:
Technology Preview Features Support Scope
- You can now hot-plug and hot-unplug virtual disks when you want to add or remove them from your virtual machine without stopping the virtual machine instance.
3.7. Known issues Copiar enlaceEnlace copiado en el portapapeles!
Updating to OpenShift Virtualization 4.8.7 causes some virtual machines (VMs) to get stuck in a live migration loop. This occurs if the
field in the VM manifest is set to a relative path.spec.volumes.containerDisk.path-
As a workaround, delete and recreate the VM manifest, setting the value of the field to an absolute path. You can then update OpenShift Virtualization.
spec.volumes.containerDisk.path
-
As a workaround, delete and recreate the VM manifest, setting the value of the
If you initially deployed OpenShift Virtualization version 2.4.z or earlier, upgrading to version 4.8 fails with the following message:
risk of data loss updating hyperconvergeds.hco.kubevirt.io: new CRD removes version v1alpha1 that is listed as a stored version on the existing CRDThis bug does not affect clusters where OpenShift Virtualization was initially deployed at version 2.5.0 or later. (BZ#1986989)
As a workaround, remove the
version from thev1alpha1custom resource definition (CRD) and resume the upgrade process:HyperConvergedOpen a proxy connection to the cluster by running the following command:
$ oc proxy &Remove the
version fromv1alpha1on the.status.storedVersionsCRD by running the following command:HyperConverged$ curl --header "Content-Type: application/json-patch+json" --request PATCH http://localhost:8001/apis/apiextensions.k8s.io/v1/customresourcedefinitions/hyperconvergeds.hco.kubevirt.io/status --data '[{"op": "replace", "path": "/status/storedVersions", "value":["v1beta1"]}]'Resume the upgrade process by running the following command:
$ curl --header "Content-Type: application/json-patch+json" --request PATCH http://localhost:8001/apis/operators.coreos.com/v1alpha1/namespaces/openshift-cnv/installplans/$(oc get installplan -n openshift-cnv | grep kubevirt-hyperconverged-operator.v4.8.0 | cut -d' ' -f1)/status --data '[{"op": "remove", "path": "/status/conditions"},{"op": "remove", "path": "/status/message"},{"op": "replace", "path": "/status/phase", "value": "Installing"}]'Kill the
process by running the following command:oc proxy$ kill $(ps -C "oc proxy" -o pid=)Optional: Monitor the upgrade status by running the following command:
$ oc get csv
- If you delete OpenShift Virtualization-provided templates in version 4.8 or later, the templates are automatically recreated by the OpenShift Virtualization Operator. However, if you delete OpenShift Virtualization-provided templates created before version 4.8, those earlier templates are not automatically recreated after deletion. As a result, any edit or update to a virtual machine referencing a deleted earlier template will fail.
If a cloning operation is initiated before the source is available to be cloned, the operation stalls indefinitely. This is because the clone authorization expires before the cloning operation starts. (BZ#1855182)
-
As a workaround, delete the object that is requesting the clone. When the source is available, recreate the
DataVolumeobject that you deleted so that the cloning operation can complete successfully.DataVolume
-
As a workaround, delete the
If your OpenShift Container Platform cluster uses OVN-Kubernetes as the default Container Network Interface (CNI) provider, you cannot attach a Linux bridge or bonding to the default interface of a host because of a change in the host network topology of OVN-Kubernetes. (BZ#1885605)
- As a workaround, you can use a secondary network interface connected to your host, or switch to the OpenShift SDN default CNI provider.
Running virtual machines that cannot be live migrated might block an OpenShift Container Platform cluster upgrade. This includes virtual machines that use hostpath-provisioner storage or SR-IOV network interfaces. (BZ#1858777)
As a workaround, you can reconfigure the virtual machines so that they can be powered off during a cluster upgrade. In the
section of the virtual machine configuration file:spec-
Remove the field. See Configuring virtual machine eviction strategy for more information on how to configure eviction strategy.
evictionStrategy: LiveMigrate -
Set the field to
runStrategy.Always
-
Remove the
Live migration fails when nodes have different CPU models. Even in cases where nodes have the same physical CPU model, differences introduced by microcode updates have the same effect. This is because the default settings trigger host CPU passthrough behavior, which is incompatible with live migration. (BZ#1760028)
As a workaround, set the default CPU model by running the following command:
NoteYou must make this change before starting the virtual machines that support live migration.
$ oc annotate --overwrite -n openshift-cnv hyperconverged kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[ { "op": "add", "path": "/spec/configuration/cpuModel", "value": "<cpu_model>"1 } ]'- 1
- Replace
<cpu_model>with the actual CPU model value. You can determine this value by runningoc describe node <node>for all nodes and looking at thecpu-model-<name>labels. Select the CPU model that is present on all of your nodes.
If you enter the wrong credentials for the RHV Manager while importing a RHV VM, the Manager might lock the admin user account because the
tries repeatedly to connect to the RHV API. (BZ#1887140)vm-import-operatorTo unlock the account, log in to the Manager and enter the following command:
$ ovirt-aaa-jdbc-tool user unlock admin
If you run OpenShift Virtualization 2.6.5 with OpenShift Container Platform 4.8, various issues occur. You can avoid these issues by upgrading OpenShift Virtualization to version 4.8.
In the web console, if you navigate to the Virtualization page and select Create → With YAML the following error message is displayed:
The server doesn't have a resource type "kind: VirtualMachine, apiVersion: kubevirt.io/v1"As a workaround, edit the
manifest so theVirtualMachineisapiVersion. For example:kubevirt.io/v1alpha3apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: annotations: ...
If you use the Customize wizard to create a VM, the following error message is displayed:
Error creating virtual machineAs a workaround, copy the manifest and create the virtual machine from the CLI.
When connecting to the VNC console by using the OpenShift Virtualization web console, the VNC console always fails to respond.
As a workaround, create the virtual machine from the CLI or upgrade to OpenShift Virtualization 4.8.
Chapter 4. Installing OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
4.1. Preparing your cluster for OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
Review this section before you install OpenShift Virtualization to ensure that your cluster meets the requirements.
You can use any installation method, including user-provisioned, installer-provisioned, or assisted installer, to deploy OpenShift Container Platform. However, the installation method and the cluster topology might affect OpenShift Virtualization functionality, such as snapshots or live migration.
FIPS mode
If you install your cluster in FIPS mode, no additional setup is required for OpenShift Virtualization.
4.1.1. Hardware and operating system requirements Copiar enlaceEnlace copiado en el portapapeles!
Review the following hardware and operating system requirements for OpenShift Virtualization.
Supported platforms
- On-premise bare metal servers
- Amazon Web Services bare metal instances
Installing OpenShift Virtualization on an AWS bare metal instance is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.
- Bare metal instances or servers offered by other cloud providers are not supported.
CPU requirements
- Supported by Red Hat Enterprise Linux (RHEL) 8
- Support for Intel 64 or AMD64 CPU extensions
- Intel VT or AMD-V hardware virtualization extensions enabled
- NX (no execute) flag enabled
Storage requirements
- Supported by OpenShift Container Platform
Operating system requirements
Red Hat Enterprise Linux CoreOS (RHCOS) installed on worker nodes
NoteRHEL worker nodes are not supported.
4.1.2. Physical resource overhead requirements Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization is an add-on to OpenShift Container Platform and imposes additional overhead that you must account for when planning a cluster. Each cluster machine must accommodate the following overhead requirements in addition to the OpenShift Container Platform requirements. Oversubscribing the physical resources in a cluster can affect performance.
The numbers noted in this documentation are based on Red Hat’s test methodology and setup. These numbers can vary based on your own individual setup and environments.
4.1.2.1. Memory overhead Copiar enlaceEnlace copiado en el portapapeles!
Calculate the memory overhead values for OpenShift Virtualization by using the equations below.
Cluster memory overhead
Memory overhead per infrastructure node ≈ 150 MiB
Memory overhead per worker node ≈ 360 MiB
Additionally, OpenShift Virtualization environment resources require a total of 2179 MiB of RAM that is spread across all infrastructure nodes.
Virtual machine memory overhead
Memory overhead per virtual machine ≈ (1.002 * requested memory) + 146 MiB \
+ 8 MiB * (number of vCPUs) \
+ 16 MiB * (number of graphics devices)
If your environment includes a Single Root I/O Virtualization (SR-IOV) network device or a Graphics Processing Unit (GPU), allocate 1 GiB additional memory overhead for each device.
4.1.2.2. CPU overhead Copiar enlaceEnlace copiado en el portapapeles!
Calculate the cluster processor overhead requirements for OpenShift Virtualization by using the equation below. The CPU overhead per virtual machine depends on your individual setup.
Cluster CPU overhead
CPU overhead for infrastructure nodes ≈ 4 cores
OpenShift Virtualization increases the overall utilization of cluster level services such as logging, routing, and monitoring. To account for this workload, ensure that nodes that host infrastructure components have capacity allocated for 4 additional cores (4000 millicores) distributed across those nodes.
CPU overhead for worker nodes ≈ 2 cores + CPU overhead per virtual machine
Each worker node that hosts virtual machines must have capacity for 2 additional cores (2000 millicores) for OpenShift Virtualization management workloads in addition to the CPUs required for virtual machine workloads.
Virtual machine CPU overhead
If dedicated CPUs are requested, there is a 1:1 impact on the cluster CPU overhead requirement. Otherwise, there are no specific rules about how many CPUs a virtual machine requires.
4.1.2.3. Storage overhead Copiar enlaceEnlace copiado en el portapapeles!
Use the guidelines below to estimate storage overhead requirements for your OpenShift Virtualization environment.
Cluster storage overhead
Aggregated storage overhead per node ≈ 10 GiB
10 GiB is the estimated on-disk storage impact for each node in the cluster when you install OpenShift Virtualization.
Virtual machine storage overhead
Storage overhead per virtual machine depends on specific requests for resource allocation within the virtual machine. The request could be for ephemeral storage on the node or storage resources hosted elsewhere in the cluster. OpenShift Virtualization does not currently allocate any additional ephemeral storage for the running container itself.
4.1.2.4. Example Copiar enlaceEnlace copiado en el portapapeles!
As a cluster administrator, if you plan to host 10 virtual machines in the cluster, each with 1 GiB of RAM and 2 vCPUs, the memory impact across the cluster is 11.68 GiB. The estimated on-disk storage impact for each node in the cluster is 10 GiB and the CPU impact for worker nodes that host virtual machine workloads is a minimum of 2 cores.
4.1.3. Object maximums Copiar enlaceEnlace copiado en el portapapeles!
You must consider the following tested object maximums when planning your cluster:
4.1.4. Restricted network environments Copiar enlaceEnlace copiado en el portapapeles!
If you install OpenShift Virtualization in a restricted environment with no internet connectivity, you must configure Operator Lifecycle Manager for restricted networks.
If you have limited internet connectivity, you can configure proxy support in Operator Lifecycle Manager to access the Red Hat-provided OperatorHub.
4.1.5. Live migration Copiar enlaceEnlace copiado en el portapapeles!
Live migration has the following requirements:
-
Shared storage with (RWX) access mode
ReadWriteMany - Sufficient RAM and network bandwidth
- Appropriate CPUs with sufficient capacity on the worker nodes. If the CPUs have different capacities, live migration might be very slow or fail.
4.1.6. Snapshots and cloning Copiar enlaceEnlace copiado en el portapapeles!
See OpenShift Virtualization storage features for snapshot and cloning requirements.
4.1.7. Cluster high-availability options Copiar enlaceEnlace copiado en el portapapeles!
You can configure one of the following high-availability (HA) options for your cluster:
Automatic high availability for installer-provisioned infrastructure (IPI) is available by deploying machine health checks.
NoteIn OpenShift Container Platform clusters installed using installer-provisioned infrastructure and with MachineHealthCheck properly configured, if a node fails the MachineHealthCheck and becomes unavailable to the cluster, it is recycled. What happens next with VMs that ran on the failed node depends on a series of conditions. See About RunStrategies for virtual machines for more detailed information about the potential outcomes and how RunStrategies affect those outcomes.
High availability for any platform is available by using either a monitoring system or a qualified human to monitor node availability. When a node is lost, shut it down and run
.oc delete node <lost_node>NoteWithout an external monitoring system or a qualified human monitoring node health, virtual machines lose high availability.
4.2. Specifying nodes for OpenShift Virtualization components Copiar enlaceEnlace copiado en el portapapeles!
Specify the nodes where you want to deploy OpenShift Virtualization Operators, workloads, and controllers by configuring node placement rules.
You can configure node placement for some components after installing OpenShift Virtualization, but there must not be virtual machines present if you want to configure node placement for workloads.
4.2.1. About node placement for virtualization components Copiar enlaceEnlace copiado en el portapapeles!
You might want to customize where OpenShift Virtualization deploys its components to ensure that:
- Virtual machines only deploy on nodes that are intended for virtualization workloads.
- Operators only deploy on infrastructure nodes.
- Certain nodes are unaffected by OpenShift Virtualization. For example, you have workloads unrelated to virtualization running on your cluster, and you want those workloads to be isolated from OpenShift Virtualization.
4.2.1.1. How to apply node placement rules to virtualization components Copiar enlaceEnlace copiado en el portapapeles!
You can specify node placement rules for a component by editing the corresponding object directly or by using the web console.
-
For the OpenShift Virtualization Operators that Operator Lifecycle Manager (OLM) deploys, edit the OLM object directly. Currently, you cannot configure node placement rules for the
Subscriptionobject by using the web console.Subscription -
For components that the OpenShift Virtualization Operators deploy, edit the object directly or configure it by using the web console during OpenShift Virtualization installation.
HyperConverged For the hostpath provisioner, edit the
object directly or configure it by using the web console.HostPathProvisionerWarningYou must schedule the hostpath provisioner and the virtualization components on the same nodes. Otherwise, virtualization pods that use the hostpath provisioner cannot run.
Depending on the object, you can use one or more of the following rule types:
nodeSelector- Allows pods to be scheduled on nodes that are labeled with the key-value pair or pairs that you specify in this field. The node must have labels that exactly match all listed pairs.
affinity- Enables you to use more expressive syntax to set rules that match nodes with pods. Affinity also allows for more nuance in how the rules are applied. For example, you can specify that a rule is a preference, rather than a hard requirement, so that pods are still scheduled if the rule is not satisfied.
tolerations- Allows pods to be scheduled on nodes that have matching taints. If a taint is applied to a node, that node only accepts pods that tolerate the taint.
4.2.1.2. Node placement in the OLM Subscription object Copiar enlaceEnlace copiado en el portapapeles!
To specify the nodes where OLM deploys the OpenShift Virtualization Operators, edit the
Subscription
spec.config
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: hco-operatorhub
namespace: openshift-cnv
spec:
source: redhat-operators
sourceNamespace: openshift-marketplace
name: kubevirt-hyperconverged
startingCSV: kubevirt-hyperconverged-operator.v4.8.7
channel: "stable"
config:
- 1
- The
configfield supportsnodeSelectorandtolerations, but it does not supportaffinity.
4.2.1.3. Node placement in the HyperConverged object Copiar enlaceEnlace copiado en el portapapeles!
To specify the nodes where OpenShift Virtualization deploys its components, you can include the
nodePlacement
nodePlacement
spec.infra
spec.workloads
apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
name: kubevirt-hyperconverged
namespace: openshift-cnv
spec:
infra:
nodePlacement:
...
workloads:
nodePlacement:
...
- 1
- The
nodePlacementfields supportnodeSelector,affinity, andtolerationsfields.
4.2.1.4. Node placement in the HostPathProvisioner object Copiar enlaceEnlace copiado en el portapapeles!
You can configure node placement rules in the
spec.workload
HostPathProvisioner
apiVersion: hostpathprovisioner.kubevirt.io/v1beta1
kind: HostPathProvisioner
metadata:
name: hostpath-provisioner
spec:
imagePullPolicy: IfNotPresent
pathConfig:
path: "</path/to/backing/directory>"
useNamingPrefix: false
workload:
- 1
- The
workloadfield supportsnodeSelector,affinity, andtolerationsfields.
4.2.2. Example manifests Copiar enlaceEnlace copiado en el portapapeles!
The following example YAML files use
nodePlacement
affinity
tolerations
4.2.2.1. Operator Lifecycle Manager Subscription object Copiar enlaceEnlace copiado en el portapapeles!
4.2.2.1.1. Example: Node placement with nodeSelector in the OLM Subscription object Copiar enlaceEnlace copiado en el portapapeles!
In this example,
nodeSelector
example.io/example-infra-key = example-infra-value
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: hco-operatorhub
namespace: openshift-cnv
spec:
source: redhat-operators
sourceNamespace: openshift-marketplace
name: kubevirt-hyperconverged
startingCSV: kubevirt-hyperconverged-operator.v4.8.7
channel: "stable"
config:
nodeSelector:
example.io/example-infra-key: example-infra-value
4.2.2.1.2. Example: Node placement with tolerations in the OLM Subscription object Copiar enlaceEnlace copiado en el portapapeles!
In this example, nodes that are reserved for OLM to deploy OpenShift Virtualization Operators are labeled with the
key=virtualization:NoSchedule
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: hco-operatorhub
namespace: openshift-cnv
spec:
source: redhat-operators
sourceNamespace: openshift-marketplace
name: kubevirt-hyperconverged
startingCSV: kubevirt-hyperconverged-operator.v4.8.7
channel: "stable"
config:
tolerations:
- key: "key"
operator: "Equal"
value: "virtualization"
effect: "NoSchedule"
4.2.2.2. HyperConverged object Copiar enlaceEnlace copiado en el portapapeles!
4.2.2.2.1. Example: Node placement with nodeSelector in the HyperConverged Cluster CR Copiar enlaceEnlace copiado en el portapapeles!
In this example,
nodeSelector
example.io/example-infra-key = example-infra-value
example.io/example-workloads-key = example-workloads-value
apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
name: kubevirt-hyperconverged
namespace: openshift-cnv
spec:
infra:
nodePlacement:
nodeSelector:
example.io/example-infra-key: example-infra-value
workloads:
nodePlacement:
nodeSelector:
example.io/example-workloads-key: example-workloads-value
4.2.2.2.2. Example: Node placement with affinity in the HyperConverged Cluster CR Copiar enlaceEnlace copiado en el portapapeles!
In this example,
affinity
example.io/example-infra-key = example-value
example.io/example-workloads-key = example-workloads-value
apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
name: kubevirt-hyperconverged
namespace: openshift-cnv
spec:
infra:
nodePlacement:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: example.io/example-infra-key
operator: In
values:
- example-infra-value
workloads:
nodePlacement:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: example.io/example-workloads-key
operator: In
values:
- example-workloads-value
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: example.io/num-cpus
operator: Gt
values:
- 8
4.2.2.2.3. Example: Node placement with tolerations in the HyperConverged Cluster CR Copiar enlaceEnlace copiado en el portapapeles!
In this example, nodes that are reserved for OpenShift Virtualization components are labeled with the
key=virtualization:NoSchedule
apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
name: kubevirt-hyperconverged
namespace: openshift-cnv
spec:
workloads:
nodePlacement:
tolerations:
- key: "key"
operator: "Equal"
value: "virtualization"
effect: "NoSchedule"
4.2.2.3. HostPathProvisioner object Copiar enlaceEnlace copiado en el portapapeles!
4.2.2.3.1. Example: Node placement with nodeSelector in the HostPathProvisioner object Copiar enlaceEnlace copiado en el portapapeles!
In this example,
nodeSelector
example.io/example-workloads-key = example-workloads-value
apiVersion: hostpathprovisioner.kubevirt.io/v1beta1
kind: HostPathProvisioner
metadata:
name: hostpath-provisioner
spec:
imagePullPolicy: IfNotPresent
pathConfig:
path: "</path/to/backing/directory>"
useNamingPrefix: false
workload:
nodeSelector:
example.io/example-workloads-key: example-workloads-value
4.3. Installing OpenShift Virtualization using the web console Copiar enlaceEnlace copiado en el portapapeles!
Install OpenShift Virtualization to add virtualization functionality to your OpenShift Container Platform cluster.
You can use the OpenShift Container Platform 4.8 web console to subscribe to and deploy the OpenShift Virtualization Operators.
4.3.1. Installing the OpenShift Virtualization Operator Copiar enlaceEnlace copiado en el portapapeles!
You can install the OpenShift Virtualization Operator from the OpenShift Container Platform web console.
Prerequisites
- Install OpenShift Container Platform 4.8 on your cluster.
-
Log in to the OpenShift Container Platform web console as a user with permissions.
cluster-admin
Procedure
- From the Administrator perspective, click Operators → OperatorHub.
- In the Filter by keyword field, type OpenShift Virtualization.
- Select the OpenShift Virtualization tile.
- Read the information about the Operator and click Install.
On the Install Operator page:
- Select stable from the list of available Update Channel options. This ensures that you install the version of OpenShift Virtualization that is compatible with your OpenShift Container Platform version.
For Installed Namespace, ensure that the Operator recommended namespace option is selected. This installs the Operator in the mandatory
namespace, which is automatically created if it does not exist.openshift-cnvWarningAttempting to install the OpenShift Virtualization Operator in a namespace other than
causes the installation to fail.openshift-cnvFor Approval Strategy, it is highly recommended that you select Automatic, which is the default value, so that OpenShift Virtualization automatically updates when a new version is available in the stable update channel.
While it is possible to select the Manual approval strategy, this is inadvisable because of the high risk that it presents to the supportability and functionality of your cluster. Only select Manual if you fully understand these risks and cannot use Automatic.
WarningBecause OpenShift Virtualization is only supported when used with the corresponding OpenShift Container Platform version, missing OpenShift Virtualization updates can cause your cluster to become unsupported.
-
Click Install to make the Operator available to the namespace.
openshift-cnv - When the Operator installs successfully, click Create HyperConverged.
- Optional: Configure Infra and Workloads node placement options for OpenShift Virtualization components.
- Click Create to launch OpenShift Virtualization.
Verification
- Navigate to the Workloads → Pods page and monitor the OpenShift Virtualization pods until they are all Running. After all the pods display the Running state, you can use OpenShift Virtualization.
4.3.2. Next steps Copiar enlaceEnlace copiado en el portapapeles!
You might want to additionally configure the following components:
- The hostpath provisioner is a local storage provisioner designed for OpenShift Virtualization. If you want to configure local storage for virtual machines, you must enable the hostpath provisioner first.
Install OpenShift Virtualization to add virtualization functionality to your OpenShift Container Platform cluster. You can subscribe to and deploy the OpenShift Virtualization Operators by using the command line to apply manifests to your cluster.
To specify the nodes where you want OpenShift Virtualization to install its components, configure node placement rules.
4.3.3. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Install OpenShift Container Platform 4.8 on your cluster.
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
4.3.4. Subscribing to the OpenShift Virtualization catalog by using the CLI Copiar enlaceEnlace copiado en el portapapeles!
Before you install OpenShift Virtualization, you must subscribe to the OpenShift Virtualization catalog. Subscribing gives the
openshift-cnv
To subscribe, configure
Namespace
OperatorGroup
Subscription
Procedure
Create a YAML file that contains the following manifest:
apiVersion: v1 kind: Namespace metadata: name: openshift-cnv --- apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: kubevirt-hyperconverged-group namespace: openshift-cnv spec: targetNamespaces: - openshift-cnv --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: hco-operatorhub namespace: openshift-cnv spec: source: redhat-operators sourceNamespace: openshift-marketplace name: kubevirt-hyperconverged startingCSV: kubevirt-hyperconverged-operator.v4.8.7 channel: "stable"1 - 1
- Using the
stablechannel ensures that you install the version of OpenShift Virtualization that is compatible with your OpenShift Container Platform version.
Create the required
,Namespace, andOperatorGroupobjects for OpenShift Virtualization by running the following command:Subscription$ oc apply -f <file name>.yaml
You can configure certificate rotation parameters in the YAML file.
4.3.5. Deploying the OpenShift Virtualization Operator by using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can deploy the OpenShift Virtualization Operator by using the
oc
Prerequisites
-
An active subscription to the OpenShift Virtualization catalog in the namespace.
openshift-cnv
Procedure
Create a YAML file that contains the following manifest:
apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec:Deploy the OpenShift Virtualization Operator by running the following command:
$ oc apply -f <file_name>.yaml
Verification
Ensure that OpenShift Virtualization deployed successfully by watching the
of the cluster service version (CSV) in thePHASEnamespace. Run the following command:openshift-cnv$ watch oc get csv -n openshift-cnvThe following output displays if deployment was successful:
Example output
NAME DISPLAY VERSION REPLACES PHASE kubevirt-hyperconverged-operator.v4.8.7 OpenShift Virtualization 4.8.7 Succeeded
4.3.6. Next steps Copiar enlaceEnlace copiado en el portapapeles!
You might want to additionally configure the following components:
- The hostpath provisioner is a local storage provisioner designed for OpenShift Virtualization. If you want to configure local storage for virtual machines, you must enable the hostpath provisioner first.
4.4. Installing the virtctl client Copiar enlaceEnlace copiado en el portapapeles!
The
virtctl
You can install the
virtctl
kubevirt-virtctl
4.4.1. Installing the virtctl client from the web console Copiar enlaceEnlace copiado en el portapapeles!
You can download the
virtctl
Prerequisites
- You must have an activated OpenShift Container Platform subscription to access the download page on the Customer Portal.
Procedure
-
Access the Customer Portal by clicking the
icon, which is in the upper-right corner of the web console, and selecting Command Line Tools.
- Ensure you have the appropriate version for your cluster selected from the Version: list.
-
Download the client for your distribution. All downloads are in
virtctlformat.tar.gz Extract the tarball. The following CLI command extracts it into the same directory as the tarball and is applicable for all distributions:
$ tar -xvf <virtctl-version-distribution.arch>.tar.gzFor Linux and macOS:
Navigate the extracted folder hierachy and make the
binary executable:virtctl$ chmod +x <virtctl-file-name>Move the
binary to a directory on your PATH.virtctlTo check your path, run:
$ echo $PATH
For Windows users:
-
Navigate the extracted folder hierarchy and double-click the executable file to install the client.
virtctl
-
Navigate the extracted folder hierarchy and double-click the
4.4.2. Enabling OpenShift Virtualization repositories Copiar enlaceEnlace copiado en el portapapeles!
Red Hat offers OpenShift Virtualization repositories for both Red Hat Enterprise Linux 8 and Red Hat Enterprise Linux 7:
-
Red Hat Enterprise Linux 8 repository:
cnv-4.8-for-rhel-8-x86_64-rpms -
Red Hat Enterprise Linux 7 repository:
rhel-7-server-cnv-4.8-rpms
The process for enabling the repository in
subscription-manager
Procedure
Enable the appropriate OpenShift Virtualization repository for your system by running the following command:
# subscription-manager repos --enable <repository>
4.4.3. Installing the virtctl client Copiar enlaceEnlace copiado en el portapapeles!
Install the
virtctl
kubevirt-virtctl
Procedure
Install the
package:kubevirt-virtctl# yum install kubevirt-virtctl
4.5. Uninstalling OpenShift Virtualization using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can uninstall OpenShift Virtualization by using the OpenShift Container Platform web console.
4.5.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have OpenShift Virtualization 4.8 installed.
You must delete all virtual machines, virtual machine instances, and data volumes.
ImportantAttempting to uninstall OpenShift Virtualization without deleting these objects results in failure.
4.5.2. Deleting the OpenShift Virtualization Operator Deployment custom resource Copiar enlaceEnlace copiado en el portapapeles!
To uninstall OpenShift Virtualization, you must first delete the OpenShift Virtualization Operator Deployment custom resource.
Prerequisites
- Create the OpenShift Virtualization Operator Deployment custom resource.
Procedure
-
From the OpenShift Container Platform web console, select from the Projects list.
openshift-cnv - Navigate to the Operators → Installed Operators page.
- Click OpenShift Virtualization.
- Click the OpenShift Virtualization Operator Deployment tab.
-
Click the Options menu
in the row containing the kubevirt-hyperconverged custom resource. In the expanded menu, click Delete HyperConverged Cluster.
- Click Delete in the confirmation window.
- Navigate to the Workloads → Pods page to verify that only the Operator pods are running.
Open a terminal window and clean up the remaining resources by running the following command:
$ oc delete apiservices v1alpha3.subresources.kubevirt.io -n openshift-cnv
4.5.3. Deleting the OpenShift Virtualization catalog subscription Copiar enlaceEnlace copiado en el portapapeles!
To finish uninstalling OpenShift Virtualization, delete the OpenShift Virtualization catalog subscription.
Prerequisites
- An active subscription to the OpenShift Virtualization catalog
Procedure
- Navigate to the Operators → OperatorHub page.
- Search for OpenShift Virtualization and then select it.
- Click Uninstall.
You can now delete the
openshift-cnv
4.5.4. Deleting a namespace using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can delete a namespace by using the OpenShift Container Platform web console.
If you do not have permissions to delete the namespace, the Delete Namespace option is not available.
Procedure
- Navigate to Administration → Namespaces.
- Locate the namespace that you want to delete in the list of namespaces.
-
On the far right side of the namespace listing, select Delete Namespace from the Options menu
.
- When the Delete Namespace pane opens, enter the name of the namespace that you want to delete in the field.
- Click Delete.
4.6. Uninstalling OpenShift Virtualization using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can uninstall OpenShift Virtualization by using the OpenShift Container Platform CLI.
4.6.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have OpenShift Virtualization 4.8 installed.
You must delete all virtual machines, virtual machine instances, and data volumes.
ImportantAttempting to uninstall OpenShift Virtualization without deleting these objects results in failure.
4.6.2. Deleting OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
You can delete OpenShift Virtualization by using the CLI.
Prerequisites
-
Install the OpenShift CLI ().
oc -
Access to a OpenShift Virtualization cluster using an account with permissions.
cluster-admin
When you delete the subscription of the OpenShift Virtualization operator in the OLM by using the CLI, the
ClusterServiceVersion
Procedure
Delete the
custom resource:HyperConverged$ oc delete HyperConverged kubevirt-hyperconverged -n openshift-cnvDelete the subscription of the OpenShift Virtualization operator in the Operator Lifecycle Manager (OLM):
$ oc delete subscription kubevirt-hyperconverged -n openshift-cnvSet the cluster service version (CSV) name for OpenShift Virtualization as an environment variable:
$ CSV_NAME=$(oc get csv -n openshift-cnv -o=jsonpath="{.items[0].metadata.name}")Delete the CSV from the OpenShift Virtualization cluster by specifying the CSV name from the previous step:
$ oc delete csv ${CSV_NAME} -n openshift-cnvOpenShift Virtualization is uninstalled when a confirmation message indicates that the CSV was deleted successfully:
Example output
clusterserviceversion.operators.coreos.com "kubevirt-hyperconverged-operator.v4.8.7" deleted
Chapter 5. Updating OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
Learn how Operator Lifecycle Manager (OLM) delivers z-stream and minor version updates for OpenShift Virtualization.
5.1. About upgrading OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
5.1.1. How OpenShift Virtualization upgrades work Copiar enlaceEnlace copiado en el portapapeles!
- Operator Lifecycle Manager (OLM) manages the lifecycle of the OpenShift Virtualization Operator. The Marketplace Operator, which is deployed during OpenShift Container Platform installation, makes external Operators available to your cluster.
- OLM provides z-stream and minor version updates for OpenShift Virtualization. Minor version updates become available when you upgrade OpenShift Container Platform to the next minor version. You cannot upgrade OpenShift Virtualization to the next minor version without first upgrading OpenShift Container Platform.
- OpenShift Virtualization subscriptions use a single update channel that is named stable. The stable channel ensures that your OpenShift Virtualization and OpenShift Container Platform versions are compatible.
If your subscription’s approval strategy is set to Automatic, the upgrade process starts as soon as a new version of the Operator is available in the stable channel. It is highly recommended to use the Automatic approval strategy to maintain a supportable environment. Each minor version of OpenShift Virtualization is only supported if you run the corresponding OpenShift Container Platform version. For example, you must run OpenShift Virtualization 4.8 on OpenShift Container Platform 4.8.
- Though it is possible to select the Manual approval strategy, this is not recommended because it risks the supportability and functionality of your cluster. With the Manual approval strategy, you must manually approve every pending update. If OpenShift Container Platform and OpenShift Virtualization updates are out of sync, your cluster becomes unsupported.
- The amount of time an update takes to complete depends on your network connection. Most automatic updates complete within fifteen minutes.
5.1.2. How OpenShift Virtualization upgrades affect your cluster Copiar enlaceEnlace copiado en el portapapeles!
Upgrading does not interrupt virtual machine workloads.
Virtual machine pods are not restarted or migrated during an upgrade. If you need to update the
pod, you must restart or live migrate the virtual machine.virt-launcherNoteEach virtual machine has a
pod that runs the virtual machine instance. Thevirt-launcherpod runs an instance ofvirt-launcher, which is used to manage the virtual machine process.libvirt
- Upgrading does not interrupt network connections.
Data volumes and their associated persistent volume claims are preserved during upgrade.
ImportantIf you have virtual machines running that cannot be live migrated, they might block an OpenShift Container Platform cluster upgrade. This includes virtual machines that use hostpath provisioner storage or SR-IOV network interfaces that have the
feature gate disabled.sriovLiveMigrationAs a workaround, you can reconfigure the virtual machines so that they can be powered off automatically during a cluster upgrade. Remove the
field and set theevictionStrategy: LiveMigratefield torunStrategy.Always
5.2. Manually approving a pending Operator update Copiar enlaceEnlace copiado en el portapapeles!
If an installed Operator has the approval strategy in its subscription set to Manual, when new updates are released in its current update channel, the update must be manually approved before installation can begin.
Prerequisites
- An Operator previously installed using Operator Lifecycle Manager (OLM).
Procedure
- In the Administrator perspective of the OpenShift Container Platform web console, navigate to Operators → Installed Operators.
- Operators that have a pending update display a status with Upgrade available. Click the name of the Operator you want to update.
- Click the Subscription tab. Any update requiring approval are displayed next to Upgrade Status. For example, it might display 1 requires approval.
- Click 1 requires approval, then click Preview Install Plan.
- Review the resources that are listed as available for update. When satisfied, click Approve.
- Navigate back to the Operators → Installed Operators page to monitor the progress of the update. When complete, the status changes to Succeeded and Up to date.
5.3. Monitoring upgrade status Copiar enlaceEnlace copiado en el portapapeles!
The best way to monitor OpenShift Virtualization upgrade status is to watch the cluster service version (CSV)
PHASE
The
PHASE
Prerequisites
-
Log in to the cluster as a user with the role.
cluster-admin -
Install the OpenShift CLI ().
oc
Procedure
Run the following command:
$ oc get csv -n openshift-cnvReview the output, checking the
field. For example:PHASEExample output
VERSION REPLACES PHASE 4.8.0 kubevirt-hyperconverged-operator.v2.6.5 Installing 4.8.1 kubevirt-hyperconverged-operator.v4.8.0 ReplacingOptional: Monitor the aggregated status of all OpenShift Virtualization component conditions by running the following command:
$ oc get hco -n openshift-cnv kubevirt-hyperconverged \ -o=jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.message}{"\n"}{end}'A successful upgrade results in the following output:
Example output
ReconcileComplete True Reconcile completed successfully Available True Reconcile completed successfully Progressing False Reconcile completed successfully Degraded False Reconcile completed successfully Upgradeable True Reconcile completed successfully
Chapter 6. Additional security privileges granted for kubevirt-controller and virt-launcher Copiar enlaceEnlace copiado en el portapapeles!
The
kubevirt-controller
6.1. Extended SELinux policies for virt-launcher pods Copiar enlaceEnlace copiado en el portapapeles!
The
container_t
-
allow process self (tun_socket (relabelfrom relabelto attach_queue)) -
allow process sysfs_t (file (write)) -
allow process hugetlbfs_t (dir (add_name create write remove_name rmdir setattr)) -
allow process hugetlbfs_t (file (create unlink))
These rules enable the following virtualization features:
- Relabel and attach queues to its own TUN sockets, which is required to support network multi-queue. Multi-queue enables network performance to scale as the number of available vCPUs increases.
-
Allows virt-launcher pods to write information to sysfs () files, which is required to enable Single Root I/O Virtualization (SR-IOV).
/sys -
Read/write entries, which is required to support huge pages. Huge pages are a method of managing large amounts of memory by increasing the memory page size.
hugetlbfs
6.2. Additional OpenShift Container Platform security context constraints and Linux capabilities for the kubevirt-controller service account Copiar enlaceEnlace copiado en el portapapeles!
Security context constraints (SCCs) control permissions for pods. These permissions include actions that a pod, a collection of containers, can perform and what resources it can access. You can use SCCs to define a set of conditions that a pod must run with to be accepted into the system.
The
kubevirt-controller
kubevirt-controller
6.2.1. Additional SCCs granted to the kubevirt-controller service account Copiar enlaceEnlace copiado en el portapapeles!
The
kubevirt-controller
The
kubevirt-controller
-
scc.AllowHostDirVolumePlugin = true
This allows virtual machines to use the hostpath volume plugin. -
scc.AllowPrivilegedContainer = false
This ensures the virt-launcher pod is not run as a privileged container. -
scc.AllowedCapabilities = []corev1.Capability{"NET_ADMIN", "NET_RAW", "SYS_NICE"}
This provides the following additional Linux capabilities,NET_ADMIN, andNET_RAW.SYS_NICE
6.2.2. Viewing the SCC and RBAC definitions for the kubevirt-controller Copiar enlaceEnlace copiado en el portapapeles!
You can view the
SecurityContextConstraints
kubevirt-controller
oc
$ oc get scc kubevirt-controller -o yaml
You can view the RBAC definition for the
kubevirt-controller
oc
$ oc get clusterrole kubevirt-controller -o yaml
Chapter 7. Using the CLI tools Copiar enlaceEnlace copiado en el portapapeles!
The two primary CLI tools used for managing resources in the cluster are:
-
The OpenShift Virtualization client
virtctl -
The OpenShift Container Platform client
oc
7.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
You must install the
virtctlclient.
7.2. Virtctl client commands Copiar enlaceEnlace copiado en el portapapeles!
The
virtctl
To view a list of
virtctl
$ virtctl help
To view a list of options that you can use with a specific command, run it with the
-h
--help
$ virtctl image-upload -h
To view a list of global command options that you can use with any
virtctl
$ virtctl options
The following table contains the
virtctl
| Command | Description |
|---|---|
|
| Start a virtual machine. |
|
| Stop a virtual machine. |
|
| Pause a virtual machine or virtual machine instance. The machine state is kept in memory. |
|
| Unpause a virtual machine or virtual machine instance. |
|
| Migrate a virtual machine. |
|
| Restart a virtual machine. |
|
| Create a service that forwards a designated port of a virtual machine or virtual machine instance and expose the service on the specified port of the node. |
|
| Connect to a serial console of a virtual machine instance. |
|
| Open a VNC (Virtual Network Client) connection to a virtual machine instance. Access the graphical console of a virtual machine instance through a VNC which requires a remote viewer on your local machine. |
|
| Display the port number and connect manually to the virtual machine instance by using any viewer through the VNC connection. |
|
| Specify a port number to run the proxy on the specified port, if that port is available. If a port number is not specified, the proxy runs on a random port. |
|
| Upload a virtual machine image to a data volume that already exists. |
|
| Upload a virtual machine image to a new data volume. |
|
| Display the client and server version information. |
|
| Display a descriptive list of
|
|
| Return a full list of file systems available on the guest machine. |
|
| Return guest agent information about the operating system. |
|
| Return a full list of logged-in users on the guest machine. |
7.3. OpenShift Container Platform client commands Copiar enlaceEnlace copiado en el portapapeles!
The OpenShift Container Platform
oc
VirtualMachine
vm
VirtualMachineInstance
vmi
You can use the
-n <namespace>
| Command | Description |
|---|---|
|
| Log in to the OpenShift Container Platform cluster as
|
|
| Display a list of objects for the specified object type in the current project. |
|
| Display details of the specific resource in the current project. |
|
| Create a resource in the current project from a file name or from stdin. |
|
| Edit a resource in the current project. |
|
| Delete a resource in the current project. |
For more comprehensive information on
oc
Chapter 8. Virtual machines Copiar enlaceEnlace copiado en el portapapeles!
8.1. Creating virtual machines Copiar enlaceEnlace copiado en el portapapeles!
Use one of these procedures to create a virtual machine:
- Quick Start guided tour
- Running the wizard
- Pasting a pre-configured YAML file with the virtual machine wizard
- Using the CLI
- Importing a VMware virtual machine or template with the virtual machine wizard
Do not create virtual machines in
openshift-*
openshift
When you create virtual machines from the web console, select a virtual machine template that is configured with a boot source. Virtual machine templates with a boot source are labeled as Available boot source or they display a customized label text. Using templates with an available boot source expedites the process of creating virtual machines.
Templates without a boot source are labeled as Boot source required. You can use these templates if you complete the steps for adding a boot source to the virtual machine.
8.1.1. Using a Quick Start to create a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
The web console provides Quick Starts with instructional guided tours for creating virtual machines. You can access the Quick Starts catalog by selecting the Help menu in the Administrator perspective to view the Quick Starts catalog. When you click on a Quick Start tile and begin the tour, the system guides you through the process.
Tasks in a Quick Start begin with selecting a Red Hat template. Then, you can add a boot source and import the operating system image. Finally, you can save the custom template and use it to create a virtual machine.
Prerequisites
- Access to the website where you can download the URL link for the operating system image.
Procedure
- In the web console, select Quick Starts from the Help menu.
- Click on a tile in the Quick Starts catalog. For example: Creating a Red Hat Linux Enterprise Linux virtual machine.
- Follow the instructions in the guided tour and complete the tasks for importing an operating system image and creating a virtual machine. The Virtual Machines tab displays the virtual machine.
8.1.2. Running the virtual machine wizard to create a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
The web console features a wizard that guides you through the process of selecting a virtual machine template and creating a virtual machine. Red Hat virtual machine templates are preconfigured with an operating system image, default settings for the operating system, flavor (CPU and memory), and workload type (server). When templates are configured with a boot source, they are labeled with a customized label text or the default label text Available boot source. These templates are then ready to be used for creating virtual machines.
You can select a template from the list of preconfigured templates, review the settings, and create a virtual machine in the Create virtual machine from template wizard. If you choose to customize your virtual machine, the wizard guides you through the General, Networking, Storage, Advanced, and Review steps. All required fields displayed by the wizard are marked by a *.
Create network interface controllers (NICs) and storage disks later and attach them to virtual machines.
Procedure
- Click Workloads → Virtualization from the side menu.
- From the Virtual Machines tab or the Templates tab, click Create and select Virtual Machine with Wizard.
- Select a template that is configured with a boot source.
- Click Next to go to the Review and create step.
- Clear the Start this virtual machine after creation checkbox if you do not want to start the virtual machine now.
- Click Create virtual machine and exit the wizard or continue with the wizard to customize the virtual machine.
Click Customize virtual machine to go to the General step.
- Optional: Edit the Name field to specify a custom name for the virtual machine.
- Optional: In the Description field, add a description.
Click Next to go to the Networking step. A
NIC is attached by default.nic0- Optional: Click Add Network Interface to create additional NICs.
-
Optional: You can remove any or all NICs by clicking the Options menu
and selecting Delete. A virtual machine does not need a NIC attached to be created. You can create NICs after the virtual machine has been created.
Click Next to go to the Storage step.
-
Optional: Click Add Disk to create additional disks. These disks can be removed by clicking the Options menu
and selecting Delete.
-
Optional: Click the Options menu
to edit the disk and save your changes.
-
Optional: Click Add Disk to create additional disks. These disks can be removed by clicking the Options menu
Click Next to go to the Advanced step to review the details for Cloud-init and configure SSH access.
NoteStatically inject an SSH key by using the custom script in cloud-init or in the wizard. This allows you to securely and remotely manage virtual machines and manage and transfer information. This step is strongly recommended to secure your VM.
- Click Next to go to the Review step and review the settings for the virtual machine.
- Click Create Virtual Machine.
Click See virtual machine details to view the Overview for this virtual machine.
The virtual machine is listed in the Virtual Machines tab.
Refer to the virtual machine wizard fields section when running the web console wizard.
8.1.2.1. Virtual machine wizard fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Parameter | Description |
|---|---|---|
| Name | The name can contain lowercase letters (
| |
| Description | Optional description field. | |
| Operating System | The operating system that is selected for the virtual machine in the template. You cannot edit this field when creating a virtual machine from a template. | |
| Boot Source | Import via URL (creates PVC) | Import content from an image available from an HTTP or HTTPS endpoint. Example: Obtaining a URL link from the web page with the operating system image. |
| Clone existing PVC (creates PVC) | Select an existent persistent volume claim available on the cluster and clone it. | |
| Import via Registry (creates PVC) | Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Example:
| |
| PXE (network boot - adds network interface) | Boot an operating system from a server on the network. Requires a PXE bootable network attachment definition. | |
| Persistent Volume Claim project | Project name that you want to use for cloning the PVC. | |
| Persistent Volume Claim name | PVC name that should apply to this virtual machine template if you are cloning an existing PVC. | |
| Mount this as a CD-ROM boot source | A CD-ROM requires an additional disk for installing the operating system. Select the checkbox to add a disk and customize it later. | |
| Flavor | Tiny, Small, Medium, Large, Custom | Presets the amount of CPU and memory in a virtual machine template with predefined values that are allocated to the virtual machine, depending on the operating system associated with that template. If you choose a default template, you can override the
|
| Workload Type Note If you choose the incorrect Workload Type, there could be performance or resource utilization issues (such as a slow UI). | Desktop | A virtual machine configuration for use on a desktop. Ideal for consumption on a small scale. Recommended for use with the web console. |
| Server | Balances performance and it is compatible with a wide range of server workloads. | |
| High-Performance | A virtual machine configuration that is optimized for high-performance workloads. | |
| Start this virtual machine after creation. | This checkbox is selected by default and the virtual machine starts running after creation. Clear the checkbox if you do not want the virtual machine to start when it is created. |
8.1.2.2. Networking fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
8.1.2.3. Storage fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Selection | Description |
|---|---|---|
| Source | Blank (creates PVC) | Create an empty disk. |
| Import via URL (creates PVC) | Import content via URL (HTTP or HTTPS endpoint). | |
| Use an existing PVC | Use a PVC that is already available in the cluster. | |
| Clone existing PVC (creates PVC) | Select an existing PVC available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Import content via container registry. | |
| Container (ephemeral) | Upload content from a container located in a registry accessible from the cluster. The container disk should be used only for read-only filesystems such as CD-ROMs or temporary virtual machines. | |
| Name | Name of the disk. The name can contain lowercase letters (
| |
| Size | Size of the disk in GiB. | |
| Type | Type of disk. Example: Disk or CD-ROM | |
| Interface | Type of disk device. Supported interfaces are virtIO, SATA, and SCSI. | |
| Storage Class | The storage class that is used to create the disk. | |
| Advanced → Volume Mode | Defines whether the persistent volume uses a formatted file system or raw block state. Default is Filesystem. | |
| Advanced → Access Mode | Access mode of the persistent volume. Supported access modes are Single User (RWO), Shared Access (RWX), and Read Only (ROX). |
Advanced storage settings
The following advanced storage settings are available for Blank, Import via URL, and Clone existing PVC disks. These parameters are optional. If you do not specify these parameters, the system uses the default values from the
kubevirt-storage-class-defaults
| Name | Parameter | Description |
|---|---|---|
| Volume Mode | Filesystem | Stores the virtual disk on a file system-based volume. |
| Block | Stores the virtual disk directly on the block volume. Only use
| |
| Access Mode | Single User (RWO) | The disk can be mounted as read/write by a single node. |
| Shared Access (RWX) | The disk can be mounted as read/write by many nodes. Note This is required for some features, such as live migration of virtual machines between nodes. | |
| Read Only (ROX) | The disk can be mounted as read-only by many nodes. |
8.1.2.4. Cloud-init fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Hostname | Sets a specific hostname for the virtual machine. |
| Authorized SSH Keys | The user’s public key that is copied to ~/.ssh/authorized_keys on the virtual machine. |
| Custom script | Replaces other options with a field in which you paste a custom cloud-init script. |
To configure storage class defaults, use storage profiles. For more information, see Customizing the storage profile.
8.1.2.5. Pasting in a pre-configured YAML file to create a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Create a virtual machine by writing or pasting a YAML configuration file. A valid
example
If your YAML configuration is invalid when you click Create, an error message indicates the parameter in which the error occurs. Only one error is shown at a time.
Navigating away from the YAML screen while editing cancels any changes to the configuration you have made.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Click Create and select Virtual Machine With YAML.
Write or paste your virtual machine configuration in the editable window.
-
Alternatively, use the virtual machine provided by default in the YAML screen.
example
-
Alternatively, use the
- Optional: Click Download to download the YAML configuration file in its present state.
- Click Create to create the virtual machine.
The virtual machine is listed in the Virtual Machines tab.
8.1.3. Using the CLI to create a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can create a virtual machine from a
virtualMachine
Procedure
Edit the
manifest for your VM. For example, the following manifest configures a Red Hat Enterprise Linux (RHEL) VM:VirtualMachineExample 8.1. Example manifest for a RHEL VM
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: labels: app: <vm_name>1 name: <vm_name> spec: dataVolumeTemplates: - apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <vm_name> spec: sourceRef: kind: DataSource name: rhel9 namespace: openshift-virtualization-os-images storage: resources: requests: storage: 30Gi running: false template: metadata: labels: kubevirt.io/domain: <vm_name> spec: domain: cpu: cores: 1 sockets: 2 threads: 1 devices: disks: - disk: bus: virtio name: rootdisk - disk: bus: virtio name: cloudinitdisk interfaces: - masquerade: {} name: default rng: {} features: smm: enabled: true firmware: bootloader: efi: {} resources: requests: memory: 8Gi evictionStrategy: LiveMigrate networks: - name: default pod: {} volumes: - dataVolume: name: <vm_name> name: rootdisk - cloudInitNoCloud: userData: |- #cloud-config user: cloud-user password: '<password>'2 chpasswd: { expire: False } name: cloudinitdiskCreate a virtual machine by using the manifest file:
$ oc create -f <vm_manifest_file>.yamlOptional: Start the virtual machine:
$ virtctl start <vm_name>
8.1.4. Virtual machine storage volume types Copiar enlaceEnlace copiado en el portapapeles!
| Storage volume type | Description |
|---|---|
| ephemeral | A local copy-on-write (COW) image that uses a network volume as a read-only backing store. The backing volume must be a PersistentVolumeClaim. The ephemeral image is created when the virtual machine starts and stores all writes locally. The ephemeral image is discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not mutated in any way. |
| persistentVolumeClaim | Attaches an available PV to a virtual machine. Attaching a PV allows for the virtual machine data to persist between sessions. Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual machine instance is the recommended method for importing existing virtual machines into OpenShift Container Platform. There are some requirements for the disk to be used within a PVC. |
| dataVolume | Data volumes build on the
Specify
|
| cloudInitNoCloud | Attaches a disk that contains the referenced cloud-init NoCloud data source, providing user data and metadata to the virtual machine. A cloud-init installation is required inside the virtual machine disk. |
| containerDisk | References an image, such as a virtual machine disk, that is stored in the container image registry. The image is pulled from the registry and attached to the virtual machine as a disk when the virtual machine is launched. A
Only RAW and QCOW2 formats are supported disk types for the container image registry. QCOW2 is recommended for reduced image size. Note A
|
| emptyDisk | Creates an additional sparse QCOW2 disk that is tied to the life-cycle of the virtual machine interface. The data survives guest-initiated reboots in the virtual machine but is discarded when the virtual machine stops or is restarted from the web console. The empty disk is used to store application dependencies and data that otherwise exceeds the limited temporary file system of an ephemeral disk. The disk capacity size must also be provided. |
8.1.5. About RunStrategies for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
A
RunStrategy
spec.runStrategy
spec.running
spec.runStrategy
spec.running
true
false
spec.running
spec.runStrategy
There are four defined RunStrategies.
Always-
A VMI is always present when a virtual machine is created. A new VMI is created if the original stops for any reason, which is the same behavior as
spec.running: true. RerunOnFailure- A VMI is re-created if the previous instance fails due to an error. The instance is not re-created if the virtual machine stops successfully, such as when it shuts down.
Manual-
The
start,stop, andrestartvirtctl client commands can be used to control the VMI’s state and existence. Halted-
No VMI is present when a virtual machine is created, which is the same behavior as
spec.running: false.
Different combinations of the
start
stop
restart
RunStrategy
The following table follows a VM’s transition from different states. The first column shows the VM’s initial
RunStrategy
RunStrategy
| Initial RunStrategy | start | stop | restart |
|---|---|---|---|
| Always | - | Halted | Always |
| RerunOnFailure | - | Halted | RerunOnFailure |
| Manual | Manual | Manual | Manual |
| Halted | Always | - | - |
In OpenShift Virtualization clusters installed using installer-provisioned infrastructure, when a node fails the MachineHealthCheck and becomes unavailable to the cluster, VMs with a RunStrategy of
Always
RerunOnFailure
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
RunStrategy: Always
template:
...
- 1
- The VMI’s current
RunStrategysetting.
8.2. Editing virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can update a virtual machine configuration using either the YAML editor in the web console or the OpenShift CLI on the command line. You can also update a subset of the parameters in the Virtual Machine Details screen.
8.2.1. Editing a virtual machine in the web console Copiar enlaceEnlace copiado en el portapapeles!
Edit select values of a virtual machine in the web console by clicking the pencil icon next to the relevant field. Other values can be edited using the CLI.
Labels and annotations are editable for both preconfigured Red Hat templates and your custom virtual machine templates. All other values are editable only for custom virtual machine templates that users have created using the Red Hat templates or the Create Virtual Machine Template wizard.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine.
- Click the Details tab.
- Click the pencil icon to make a field editable.
- Make the relevant changes and click Save.
If the virtual machine is running, changes to Boot Order or Flavor will not take effect until you restart the virtual machine.
You can view pending changes by clicking View Pending Changes on the right side of the relevant field. The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
8.2.2. Editing a virtual machine YAML configuration using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can edit the YAML configuration of a virtual machine in the web console. Some parameters cannot be modified. If you click Save with an invalid configuration, an error message indicates the parameter that cannot be changed.
If you edit the YAML configuration while the virtual machine is running, changes will not take effect until you restart the virtual machine.
Navigating away from the YAML screen while editing cancels any changes to the configuration you have made.
Procedure
- Click Workloads → Virtualization from the side menu.
- Select a virtual machine.
- Click the YAML tab to display the editable configuration.
- Optional: You can click Download to download the YAML file locally in its current state.
- Edit the file and click Save.
A confirmation message shows that the modification has been successful and includes the updated version number for the object.
8.2.3. Editing a virtual machine YAML configuration using the CLI Copiar enlaceEnlace copiado en el portapapeles!
Use this procedure to edit a virtual machine YAML configuration using the CLI.
Prerequisites
- You configured a virtual machine with a YAML object configuration file.
-
You installed the CLI.
oc
Procedure
Run the following command to update the virtual machine configuration:
$ oc edit <object_type> <object_ID>- Open the object configuration.
- Edit the YAML.
If you edit a running virtual machine, you need to do one of the following:
- Restart the virtual machine.
Run the following command for the new configuration to take effect:
$ oc apply <object_type> <object_ID>
8.2.4. Adding a virtual disk to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Use this procedure to add a virtual disk to a virtual machine.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Disks tab.
In the Add Disk window, specify the Source, Name, Size, Type, Interface, and Storage Class.
-
Optional: In the Advanced list, specify the Volume Mode and Access Mode for the virtual disk. If you do not specify these parameters, the system uses the default values from the config map.
kubevirt-storage-class-defaults
-
Optional: In the Advanced list, specify the Volume Mode and Access Mode for the virtual disk. If you do not specify these parameters, the system uses the default values from the
- Click Add.
If the virtual machine is running, the new disk is in the pending restart state and will not be attached until you restart the virtual machine.
The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
To configure storage class defaults, use storage profiles. For more information, see Customizing the storage profile.
8.2.4.1. Storage fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Selection | Description |
|---|---|---|
| Source | Blank (creates PVC) | Create an empty disk. |
| Import via URL (creates PVC) | Import content via URL (HTTP or HTTPS endpoint). | |
| Use an existing PVC | Use a PVC that is already available in the cluster. | |
| Clone existing PVC (creates PVC) | Select an existing PVC available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Import content via container registry. | |
| Container (ephemeral) | Upload content from a container located in a registry accessible from the cluster. The container disk should be used only for read-only filesystems such as CD-ROMs or temporary virtual machines. | |
| Name | Name of the disk. The name can contain lowercase letters (
| |
| Size | Size of the disk in GiB. | |
| Type | Type of disk. Example: Disk or CD-ROM | |
| Interface | Type of disk device. Supported interfaces are virtIO, SATA, and SCSI. | |
| Storage Class | The storage class that is used to create the disk. | |
| Advanced → Volume Mode | Defines whether the persistent volume uses a formatted file system or raw block state. Default is Filesystem. | |
| Advanced → Access Mode | Access mode of the persistent volume. Supported access modes are Single User (RWO), Shared Access (RWX), and Read Only (ROX). |
Advanced storage settings
The following advanced storage settings are available for Blank, Import via URL, and Clone existing PVC disks. These parameters are optional. If you do not specify these parameters, the system uses the default values from the
kubevirt-storage-class-defaults
| Name | Parameter | Description |
|---|---|---|
| Volume Mode | Filesystem | Stores the virtual disk on a file system-based volume. |
| Block | Stores the virtual disk directly on the block volume. Only use
| |
| Access Mode | Single User (RWO) | The disk can be mounted as read/write by a single node. |
| Shared Access (RWX) | The disk can be mounted as read/write by many nodes. Note This is required for some features, such as live migration of virtual machines between nodes. | |
| Read Only (ROX) | The disk can be mounted as read-only by many nodes. |
8.2.5. Adding a network interface to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Use this procedure to add a network interface to a virtual machine.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Network Interfaces tab.
- Click Add Network Interface.
- In the Add Network Interface window, specify the Name, Model, Network, Type, and MAC Address of the network interface.
- Click Add.
If the virtual machine is running, the new network interface is in the pending restart state and changes will not take effect until you restart the virtual machine.
The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
8.2.5.1. Networking fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
8.2.6. Editing CD-ROMs for Virtual Machines Copiar enlaceEnlace copiado en el portapapeles!
Use the following procedure to edit CD-ROMs for virtual machines.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Disks tab.
-
Click the Options menu
for the CD-ROM that you want to edit and select Edit.
- In the Edit CD-ROM window, edit the fields: Source, Persistent Volume Claim, Name, Type, and Interface.
- Click Save.
8.3. Editing boot order Copiar enlaceEnlace copiado en el portapapeles!
You can update the values for a boot order list by using the web console or the CLI.
With Boot Order in the Virtual Machine Overview page, you can:
- Select a disk or network interface controller (NIC) and add it to the boot order list.
- Edit the order of the disks or NICs in the boot order list.
- Remove a disk or NIC from the boot order list, and return it back to the inventory of bootable sources.
8.3.1. Adding items to a boot order list in the web console Copiar enlaceEnlace copiado en el portapapeles!
Add items to a boot order list by using the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Details tab.
- Click the pencil icon that is located on the right side of Boot Order. If a YAML configuration does not exist, or if this is the first time that you are creating a boot order list, the following message displays: No resource selected. VM will attempt to boot from disks by order of appearance in YAML file.
- Click Add Source and select a bootable disk or network interface controller (NIC) for the virtual machine.
- Add any additional disks or NICs to the boot order list.
- Click Save.
If the virtual machine is running, changes to Boot Order will not take effect until you restart the virtual machine.
You can view pending changes by clicking View Pending Changes on the right side of the Boot Order field. The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
8.3.2. Editing a boot order list in the web console Copiar enlaceEnlace copiado en el portapapeles!
Edit the boot order list in the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Details tab.
- Click the pencil icon that is located on the right side of Boot Order.
Choose the appropriate method to move the item in the boot order list:
- If you do not use a screen reader, hover over the arrow icon next to the item that you want to move, drag the item up or down, and drop it in a location of your choice.
- If you use a screen reader, press the Up Arrow key or Down Arrow key to move the item in the boot order list. Then, press the Tab key to drop the item in a location of your choice.
- Click Save.
If the virtual machine is running, changes to the boot order list will not take effect until you restart the virtual machine.
You can view pending changes by clicking View Pending Changes on the right side of the Boot Order field. The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
8.3.3. Editing a boot order list in the YAML configuration file Copiar enlaceEnlace copiado en el portapapeles!
Edit the boot order list in a YAML configuration file by using the CLI.
Procedure
Open the YAML configuration file for the virtual machine by running the following command:
$ oc edit vm exampleEdit the YAML file and modify the values for the boot order associated with a disk or network interface controller (NIC). For example:
disks: - bootOrder: 11 disk: bus: virtio name: containerdisk - disk: bus: virtio name: cloudinitdisk - cdrom: bus: virtio name: cd-drive-1 interfaces: - boot Order: 22 macAddress: '02:96:c4:00:00' masquerade: {} name: default- Save the YAML file.
- Click reload the content to apply the updated boot order values from the YAML file to the boot order list in the web console.
8.3.4. Removing items from a boot order list in the web console Copiar enlaceEnlace copiado en el portapapeles!
Remove items from a boot order list by using the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Details tab.
- Click the pencil icon that is located on the right side of Boot Order.
-
Click the Remove icon
next to the item. The item is removed from the boot order list and saved in the list of available boot sources. If you remove all items from the boot order list, the following message displays: No resource selected. VM will attempt to boot from disks by order of appearance in YAML file.
If the virtual machine is running, changes to Boot Order will not take effect until you restart the virtual machine.
You can view pending changes by clicking View Pending Changes on the right side of the Boot Order field. The Pending Changes banner at the top of the page displays a list of all changes that will be applied when the virtual machine restarts.
8.4. Deleting virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can delete a virtual machine from the web console or by using the
oc
8.4.1. Deleting a virtual machine using the web console Copiar enlaceEnlace copiado en el portapapeles!
Deleting a virtual machine permanently removes it from the cluster.
When you delete a virtual machine, the data volume it uses is automatically deleted.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
Click the Options menu
of the virtual machine that you want to delete and select Delete Virtual Machine.
- Alternatively, click the virtual machine name to open the Virtual Machine Overview screen and click Actions → Delete Virtual Machine.
- In the confirmation pop-up window, click Delete to permanently delete the virtual machine.
8.4.2. Deleting a virtual machine by using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can delete a virtual machine by using the
oc
oc
When you delete a virtual machine, the data volume it uses is automatically deleted.
Prerequisites
- Identify the name of the virtual machine that you want to delete.
Procedure
Delete the virtual machine by running the following command:
$ oc delete vm <vm_name>NoteThis command only deletes objects that exist in the current project. Specify the
option if the object you want to delete is in a different project or namespace.-n <project_name>
8.5. Managing virtual machine instances Copiar enlaceEnlace copiado en el portapapeles!
If you have standalone virtual machine instances (VMIs) that were created independently outside of the OpenShift Virtualization environment, you can manage them by using the web console or the command-line interface (CLI).
8.5.1. About virtual machine instances Copiar enlaceEnlace copiado en el portapapeles!
A virtual machine instance (VMI) is a representation of a running virtual machine (VM). When a VMI is owned by a VM or by another object, you manage it through its owner in the web console or by using the
oc
A standalone VMI is created and started independently with a script, through automation, or by using other methods in the CLI. In your environment, you might have standalone VMIs that were developed and started outside of the OpenShift Virtualization environment. You can continue to manage those standalone VMIs by using the CLI. You can also use the web console for specific tasks associated with standalone VMIs:
- List standalone VMIs and their details.
- Edit labels and annotations for a standalone VMI.
- Delete a standalone VMI.
When you delete a VM, the associated VMI is automatically deleted. You delete a standalone VMI directly because it is not owned by VMs or other objects.
Before you uninstall OpenShift Virtualization, list and view the standalone VMIs by using the CLI or the web console. Then, delete any outstanding VMIs.
8.5.2. Listing all virtual machine instances using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can list all virtual machine instances (VMIs) in your cluster, including standalone VMIs and those owned by virtual machines, by using the
oc
Procedure
List all VMIs by running the following command:
$ oc get vmis
8.5.3. Listing standalone virtual machine instances using the web console Copiar enlaceEnlace copiado en el portapapeles!
Using the web console, you can list and view standalone virtual machine instances (VMIs) in your cluster that are not owned by virtual machines (VMs).
VMIs that are owned by VMs or other objects are not displayed in the web console. The web console displays only standalone VMIs. If you want to list all VMIs in your cluster, you must use the CLI.
Procedure
- Click Workloads → Virtualization from the side menu. A list of VMs and standalone VMIs displays. You can identify standalone VMIs by the dark colored badges that display next to the virtual machine instance names.
8.5.4. Editing a standalone virtual machine instance using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can edit annotations and labels for a standalone virtual machine instance (VMI) using the web console. Other items displayed in the Details page for a standalone VMI are not editable.
Procedure
- Click Workloads → Virtualization from the side menu. A list of virtual machines (VMs) and standalone VMIs displays.
- Click the name of a standalone VMI to open the Virtual Machine Instance Overview screen.
- Click the Details tab.
- Click the pencil icon that is located on the right side of Annotations.
- Make the relevant changes and click Save.
To edit labels for a standalone VMI, click Actions and select Edit Labels. Make the relevant changes and click Save.
8.5.5. Deleting a standalone virtual machine instance using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can delete a standalone virtual machine instance (VMI) by using the
oc
Prerequisites
- Identify the name of the VMI that you want to delete.
Procedure
Delete the VMI by running the following command:
$ oc delete vmi <vmi_name>
8.5.6. Deleting a standalone virtual machine instance using the web console Copiar enlaceEnlace copiado en el portapapeles!
Delete a standalone virtual machine instance (VMI) from the web console.
Procedure
- In the OpenShift Container Platform web console, click Workloads → Virtualization from the side menu.
Click the ⋮ button of the standalone virtual machine instance (VMI) that you want to delete and select Delete Virtual Machine Instance.
- Alternatively, click the name of the standalone VMI. The Virtual Machine Instance Overview page displays.
- Select Actions → Delete Virtual Machine Instance.
- In the confirmation pop-up window, click Delete to permanently delete the standalone VMI.
8.6. Controlling virtual machine states Copiar enlaceEnlace copiado en el portapapeles!
You can stop, start, restart, and unpause virtual machines from the web console.
To control virtual machines from the command-line interface (CLI), use the virtctl client.
8.6.1. Starting a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can start a virtual machine from the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Find the row that contains the virtual machine that you want to start.
Navigate to the appropriate menu for your use case:
To stay on this page, where you can perform actions on multiple virtual machines:
-
Click the Options menu
located at the far right end of the row.
-
Click the Options menu
To view comprehensive information about the selected virtual machine before you start it:
- Access the Virtual Machine Overview screen by clicking the name of the virtual machine.
- Click Actions.
- Select Start Virtual Machine.
- In the confirmation window, click Start to start the virtual machine.
When you start virtual machine that is provisioned from a
URL
8.6.2. Restarting a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can restart a running virtual machine from the web console.
To avoid errors, do not restart a virtual machine while it has a status of Importing.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Find the row that contains the virtual machine that you want to restart.
Navigate to the appropriate menu for your use case:
To stay on this page, where you can perform actions on multiple virtual machines:
-
Click the Options menu
located at the far right end of the row.
-
Click the Options menu
To view comprehensive information about the selected virtual machine before you restart it:
- Access the Virtual Machine Overview screen by clicking the name of the virtual machine.
- Click Actions.
- Select Restart Virtual Machine.
- In the confirmation window, click Restart to restart the virtual machine.
8.6.3. Stopping a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can stop a virtual machine from the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Find the row that contains the virtual machine that you want to stop.
Navigate to the appropriate menu for your use case:
To stay on this page, where you can perform actions on multiple virtual machines:
-
Click the Options menu
located at the far right end of the row.
-
Click the Options menu
To view comprehensive information about the selected virtual machine before you stop it:
- Access the Virtual Machine Overview screen by clicking the name of the virtual machine.
- Click Actions.
- Select Stop Virtual Machine.
- In the confirmation window, click Stop to stop the virtual machine.
8.6.4. Unpausing a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can unpause a paused virtual machine from the web console.
Prerequisites
At least one of your virtual machines must have a status of Paused.
NoteYou can pause virtual machines by using the
client.virtctl
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Find the row that contains the virtual machine that you want to unpause.
Navigate to the appropriate menu for your use case:
To stay on this page, where you can perform actions on multiple virtual machines:
- In the Status column, click Paused.
To view comprehensive information about the selected virtual machine before you unpause it:
- Access the Virtual Machine Overview screen by clicking the name of the virtual machine.
- Click the pencil icon that is located on the right side of Status.
- In the confirmation window, click Unpause to unpause the virtual machine.
8.7. Accessing virtual machine consoles Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization provides different virtual machine consoles that you can use to accomplish different product tasks. You can access these consoles through the OpenShift Container Platform web console and by using CLI commands.
8.7.1. Accessing virtual machine consoles in the OpenShift Container Platform web console Copiar enlaceEnlace copiado en el portapapeles!
You can connect to virtual machines by using the serial console or the VNC console in the OpenShift Container Platform web console.
You can connect to Windows virtual machines by using the desktop viewer console, which uses RDP (remote desktop protocol), in the OpenShift Container Platform web console.
8.7.1.1. Connecting to the serial console Copiar enlaceEnlace copiado en el portapapeles!
Connect to the serial console of a running virtual machine from the Console tab in the Virtual Machine Overview screen of the web console.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview page.
- Click Console. The VNC console opens by default.
- Select Disconnect before switching to ensure that only one console session is open at a time. Otherwise, the VNC console session remains active in the background.
- Click the VNC Console drop-down list and select Serial Console.
- Click Disconnect to end the console session.
- Optional: Open the serial console in a separate window by clicking Open Console in New Window.
8.7.1.2. Connecting to the VNC console Copiar enlaceEnlace copiado en el portapapeles!
Connect to the VNC console of a running virtual machine from the Console tab in the Virtual Machine Overview screen of the web console.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview page.
- Click the Console tab. The VNC console opens by default.
- Optional: Open the VNC console in a separate window by clicking Open Console in New Window.
- Optional: Send key combinations to the virtual machine by clicking Send Key.
- Click outside the console window and then click Disconnect to end the session.
8.7.1.3. Connecting to a Windows virtual machine with RDP Copiar enlaceEnlace copiado en el portapapeles!
The desktop viewer console, which utilizes the Remote Desktop Protocol (RDP), provides a better console experience for connecting to Windows virtual machines.
To connect to a Windows virtual machine with RDP, download the
console.rdp
Prerequisites
-
A running Windows virtual machine with the QEMU guest agent installed. The is included in the VirtIO drivers.
qemu-guest-agent - A layer-2 NIC attached to the virtual machine.
- An RDP client installed on a machine on the same network as the Windows virtual machine.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a Windows virtual machine to open the Virtual Machine Overview screen.
- Click the Console tab.
- In the Console list, select Desktop Viewer.
- In the Network Interface list, select the layer-2 NIC.
-
Click Launch Remote Desktop to download the file.
console.rdp Open an RDP client and reference the
file. For example, using remmina:console.rdp$ remmina --connect /path/to/console.rdp- Enter the Administrator user name and password to connect to the Windows virtual machine.
8.7.1.4. Copying the SSH command from the web console Copiar enlaceEnlace copiado en el portapapeles!
Copy the command to access a running virtual machine (VM) via SSH from the Actions list in the web console.
Procedure
- In the OpenShift Container Platform console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview page.
-
From the Actions list, select Copy SSH Command. You can now paste this command onto the OpenShift CLI ().
oc
8.7.2. Accessing virtual machine consoles by using CLI commands Copiar enlaceEnlace copiado en el portapapeles!
8.7.2.1. Accessing a virtual machine instance via SSH Copiar enlaceEnlace copiado en el portapapeles!
You can use SSH to access a virtual machine (VM) after you expose port 22 on it.
The
virtctl expose
fedora-vm-ssh
<fedora-vm>
Prerequisites
- You must be in the same project as the VMI.
-
The VMI you want to access must be connected to the default pod network by using the binding method.
masquerade - The VMI you want to access must be running.
-
Install the OpenShift CLI ().
oc
Procedure
Run the following command to create the
service:fedora-vm-ssh$ virtctl expose vm <fedora-vm> --port=22 --name=fedora-vm-ssh --type=NodePort1 - 1
<fedora-vm>is the name of the VM that you run thefedora-vm-sshservice on.
Check the service to find out which port the service acquired:
$ oc get svc
Example output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
fedora-vm-ssh NodePort 127.0.0.1 <none> 22:32551/TCP 6s
+ In this example, the service acquired the
32551
Log in to the VMI via SSH. Use the
of any of the cluster nodes and the port that you found in the previous step:ipAddress$ ssh username@<node_IP_address> -p 32551
8.7.2.2. Accessing a virtual machine via SSH with YAML configurations Copiar enlaceEnlace copiado en el portapapeles!
You can enable an SSH connection to a virtual machine (VM) without the need to run the
virtctl expose
The following examples show the configurations for the VM’s YAML file and the service YAML file.
Prerequisites
-
Install the OpenShift CLI ().
oc -
Create a namespace for the VM’s YAML file by using the command and specifying a name for the namespace.
oc create namespace
Procedure
In the YAML file for the VM, add the label and a value for exposing the service for SSH connections. Enable the
feature for the interface:masqueradeExample
VirtualMachinedefinition... apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: namespace: ssh-ns1 name: vm-ssh spec: running: false template: metadata: labels: kubevirt.io/vm: vm-ssh special: vm-ssh2 spec: domain: devices: disks: - disk: bus: virtio name: containerdisk - disk: bus: virtio name: cloudinitdisk interfaces: - masquerade: {}3 name: testmasquerade4 rng: {} machine: type: "" resources: requests: memory: 1024M networks: - name: testmasquerade pod: {} volumes: - name: containerdisk containerDisk: image: kubevirt/fedora-cloud-container-disk-demo - name: cloudinitdisk cloudInitNoCloud: userData: | #!/bin/bash echo "fedora" | passwd fedora --stdin ...- 1
- Name of the namespace created by the
oc create namespacecommand. - 2
- Label used by the service to identify the virtual machine instances that are enabled for SSH traffic connections. The label can be any
key:valuepair that is added as alabelto this YAML file and as aselectorin the service YAML file. - 3
- The interface type is
masquerade. - 4
- The name of this interface is
testmasquerade.
Create the VM:
$ oc create -f <path_for_the_VM_YAML_file>Start the VM:
$ virtctl start vm-sshIn the YAML file for the service, specify the service name, port number, and the target port.
Example
Servicedefinition... apiVersion: v1 kind: Service metadata: name: svc-ssh1 namespace: ssh-ns2 spec: ports: - targetPort: 223 protocol: TCP port: 27017 selector: special: vm-ssh4 type: NodePort ...Create the service:
$ oc create -f <path_for_the_service_YAML_file>Verify that the VM is running:
$ oc get vmiExample output
NAME AGE PHASE IP NODENAME vm-ssh 6s Running 10.244.196.152 node01Check the service to find out which port the service acquired:
$ oc get svcExample output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc-ssh NodePort 10.106.236.208 <none> 27017:30093/TCP 22sIn this example, the service acquired the port number 30093.
Run the following command to obtain the IP address for the node:
$ oc get node <node_name> -o wideExample output
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP node01 Ready worker 6d22h v1.20.0+5f82cdb 192.168.55.101 <none>Log in to the VM via SSH by specifying the IP address of the node where the VM is running and the port number. Use the port number displayed by the
command and the IP address of the node displayed by theoc get svccommand. The following example shows theoc get nodecommand with the username, node’s IP address, and the port number:ssh$ ssh fedora@192.168.55.101 -p 30093
8.7.2.3. Accessing the serial console of a virtual machine instance Copiar enlaceEnlace copiado en el portapapeles!
The
virtctl console
Prerequisites
-
The package must be installed.
virt-viewer - The virtual machine instance you want to access must be running.
Procedure
Connect to the serial console with
:virtctl$ virtctl console <VMI>
8.7.2.4. Accessing the graphical console of a virtual machine instances with VNC Copiar enlaceEnlace copiado en el portapapeles!
The
virtctl
remote-viewer
virt-viewer
Prerequisites
-
The package must be installed.
virt-viewer - The virtual machine instance you want to access must be running.
If you use
virtctl
Procedure
Connect to the graphical interface with the
utility:virtctl$ virtctl vnc <VMI>If the command failed, try using the
flag to collect troubleshooting information:-v$ virtctl vnc <VMI> -v 4
8.7.2.5. Connecting to a Windows virtual machine with an RDP console Copiar enlaceEnlace copiado en el portapapeles!
The Remote Desktop Protocol (RDP) provides a better console experience for connecting to Windows virtual machines.
To connect to a Windows virtual machine with RDP, specify the IP address of the attached L2 NIC to your RDP client.
Prerequisites
-
A running Windows virtual machine with the QEMU guest agent installed. The is included in the VirtIO drivers.
qemu-guest-agent - A layer 2 NIC attached to the virtual machine.
- An RDP client installed on a machine on the same network as the Windows virtual machine.
Procedure
Log in to the OpenShift Virtualization cluster through the
CLI tool as a user with an access token.oc$ oc login -u <user> https://<cluster.example.com>:8443Use
to display the configuration of the running Windows virtual machine.oc describe vmi$ oc describe vmi <windows-vmi-name>Example output
... spec: networks: - name: default pod: {} - multus: networkName: cnv-bridge name: bridge-net ... status: interfaces: - interfaceName: eth0 ipAddress: 198.51.100.0/24 ipAddresses: 198.51.100.0/24 mac: a0:36:9f:0f:b1:70 name: default - interfaceName: eth1 ipAddress: 192.0.2.0/24 ipAddresses: 192.0.2.0/24 2001:db8::/32 mac: 00:17:a4:77:77:25 name: bridge-net ...-
Identify and copy the IP address of the layer 2 network interface. This is in the above example, or
192.0.2.0if you prefer IPv6.2001:db8:: - Open an RDP client and use the IP address copied in the previous step for the connection.
- Enter the Administrator user name and password to connect to the Windows virtual machine.
8.8. Triggering virtual machine failover by resolving a failed node Copiar enlaceEnlace copiado en el portapapeles!
If a node fails and machine health checks are not deployed on your cluster, virtual machines (VMs) with
RunStrategy: Always
Node
If you installed your cluster by using installer-provisioned infrastructure and you properly configured machine health checks:
- Failed nodes are automatically recycled.
-
Virtual machines with
RunStrategyset toorAlwaysare automatically scheduled on healthy nodes.RerunOnFailure
8.8.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
A node where a virtual machine was running has the condition.
NotReady -
The virtual machine that was running on the failed node has set to
RunStrategy.Always -
You have installed the OpenShift CLI ().
oc
8.8.2. Deleting nodes from a bare metal cluster Copiar enlaceEnlace copiado en el portapapeles!
When you delete a node using the CLI, the node object is deleted in Kubernetes, but the pods that exist on the node are not deleted. Any bare pods not backed by a replication controller become inaccessible to OpenShift Container Platform. Pods backed by replication controllers are rescheduled to other available nodes. You must delete local manifest pods.
Procedure
Delete a node from an OpenShift Container Platform cluster running on bare metal by completing the following steps:
Mark the node as unschedulable:
$ oc adm cordon <node_name>Drain all pods on the node:
$ oc adm drain <node_name> --force=trueThis step might fail if the node is offline or unresponsive. Even if the node does not respond, it might still be running a workload that writes to shared storage. To avoid data corruption, power down the physical hardware before you proceed.
Delete the node from the cluster:
$ oc delete node <node_name>Although the node object is now deleted from the cluster, it can still rejoin the cluster after reboot or if the kubelet service is restarted. To permanently delete the node and all its data, you must decommission the node.
- If you powered down the physical hardware, turn it back on so that the node can rejoin the cluster.
8.8.3. Verifying virtual machine failover Copiar enlaceEnlace copiado en el portapapeles!
After all resources are terminated on the unhealthy node, a new virtual machine instance (VMI) is automatically created on a healthy node for each relocated VM. To confirm that the VMI was created, view all VMIs by using the
oc
8.8.3.1. Listing all virtual machine instances using the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can list all virtual machine instances (VMIs) in your cluster, including standalone VMIs and those owned by virtual machines, by using the
oc
Procedure
List all VMIs by running the following command:
$ oc get vmis
8.9. Installing the QEMU guest agent on virtual machines Copiar enlaceEnlace copiado en el portapapeles!
The QEMU guest agent is a daemon that runs on the virtual machine and passes information to the host about the virtual machine, users, file systems, and secondary networks.
8.9.1. Installing QEMU guest agent on a Linux virtual machine Copiar enlaceEnlace copiado en el portapapeles!
The
qemu-guest-agent
Procedure
- Access the virtual machine command line through one of the consoles or by SSH.
Install the QEMU guest agent on the virtual machine:
$ yum install -y qemu-guest-agentEnsure the service is persistent and start it:
$ systemctl enable --now qemu-guest-agent
You can also install and start the QEMU guest agent by using the custom script field in the cloud-init section of the wizard when creating either virtual machines or virtual machines templates in the web console.
8.9.2. Installing QEMU guest agent on a Windows virtual machine Copiar enlaceEnlace copiado en el portapapeles!
For Windows virtual machines, the QEMU guest agent is included in the VirtIO drivers, which can be installed using one of the following procedures:
8.9.2.1. Installing VirtIO drivers on an existing Windows virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Install the VirtIO drivers from the attached SATA CD drive to an existing Windows virtual machine.
This procedure uses a generic approach to adding drivers to Windows. The process might differ slightly between versions of Windows. See the installation documentation for your version of Windows for specific installation steps.
Procedure
- Start the virtual machine and connect to a graphical console.
- Log in to a Windows user session.
Open Device Manager and expand Other devices to list any Unknown device.
-
Open the to identify the unknown device. Right-click the device and select Properties.
Device Properties - Click the Details tab and select Hardware Ids in the Property list.
- Compare the Value for the Hardware Ids with the supported VirtIO drivers.
-
Open the
- Right-click the device and select Update Driver Software.
- Click Browse my computer for driver software and browse to the attached SATA CD drive, where the VirtIO drivers are located. The drivers are arranged hierarchically according to their driver type, operating system, and CPU architecture.
- Click Next to install the driver.
- Repeat this process for all the necessary VirtIO drivers.
- After the driver installs, click Close to close the window.
- Reboot the virtual machine to complete the driver installation.
8.9.2.2. Installing VirtIO drivers during Windows installation Copiar enlaceEnlace copiado en el portapapeles!
Install the VirtIO drivers from the attached SATA CD driver during Windows installation.
This procedure uses a generic approach to the Windows installation and the installation method might differ between versions of Windows. See the documentation for the version of Windows that you are installing.
Procedure
- Start the virtual machine and connect to a graphical console.
- Begin the Windows installation process.
- Select the Advanced installation.
-
The storage destination will not be recognized until the driver is loaded. Click .
Load driver - The drivers are attached as a SATA CD drive. Click OK and browse the CD drive for the storage driver to load. The drivers are arranged hierarchically according to their driver type, operating system, and CPU architecture.
- Repeat the previous two steps for all required drivers.
- Complete the Windows installation.
8.10. Viewing the QEMU guest agent information for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
When the QEMU guest agent runs on the virtual machine, you can use the web console to view information about the virtual machine, users, file systems, and secondary networks.
8.10.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Install the QEMU guest agent on the virtual machine.
8.10.2. About the QEMU guest agent information in the web console Copiar enlaceEnlace copiado en el portapapeles!
When the QEMU guest agent is installed, the Details pane within the Virtual Machine Overview tab and the Details tab display information about the hostname, operating system, time zone, and logged in users.
The Virtual Machine Overview shows information about the guest operating system installed on the virtual machine. The Details tab displays a table with information for logged in users. The Disks tab displays a table with information for file systems.
If the QEMU guest agent is not installed, the Virtual Machine Overview tab and the Details tab display information about the operating system that was specified when the virtual machine was created.
8.10.3. Viewing the QEMU guest agent information in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can use the web console to view information for virtual machines that is passed by the QEMU guest agent to the host.
Procedure
- Click Workloads → Virtual Machines from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine name to open the Virtual Machine Overview screen and view the Details pane.
- Click Logged in users to view the Details tab that shows information for users.
- Click the Disks tab to view information about the file systems.
8.11. Managing config maps, secrets, and service accounts in virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can use secrets, config maps, and service accounts to pass configuration data to virtual machines. For example, you can:
- Give a virtual machine access to a service that requires credentials by adding a secret to the virtual machine.
- Store non-confidential configuration data in a config map so that a pod or another object can consume the data.
- Allow a component to access the API server by associating a service account with that component.
OpenShift Virtualization exposes secrets, config maps, and service accounts as virtual machine disks so that you can use them across platforms without additional overhead.
8.11.1. Adding a secret, config map, or service account to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Add a secret, config map, or service account to a virtual machine by using the OpenShift Container Platform web console.
Prerequisites
- The secret, config map, or service account that you want to add must exist in the same namespace as the target virtual machine.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Environment tab.
- Click Select a resource and select a secret, config map, or service account from the list. A six character serial number is automatically generated for the selected resource.
- Click Save.
- Optional. Add another object by clicking Add Config Map, Secret or Service Account.
- You can reset the form to the last saved state by clicking Reload.
- The Environment resources are added to the virtual machine as disks. You can mount the secret, config map, or service account as you would mount any other disk.
- If the virtual machine is running, changes will not take effect until you restart the virtual machine. The newly added resources are marked as pending changes for both the Environment and Disks tab in the Pending Changes banner at the top of the page.
Verification
- From the Virtual Machine Overview page, click the Disks tab.
- Check to ensure that the secret, config map, or service account is included in the list of disks.
Optional. Choose the appropriate method to apply your changes:
- If the virtual machine is running, restart the virtual machine by clicking Actions → Restart Virtual Machine.
- If the virtual machine is stopped, start the virtual machine by clicking Actions → Start Virtual Machine.
You can now mount the secret, config map, or service account as you would mount any other disk.
8.11.2. Removing a secret, config map, or service account from a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Remove a secret, config map, or service account from a virtual machine by using the OpenShift Container Platform web console.
Prerequisites
- You must have at least one secret, config map, or service account that is attached to a virtual machine.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Environment tab.
-
Find the item that you want to delete in the list, and click Remove
on the right side of the item.
- Click Save.
You can reset the form to the last saved state by clicking Reload.
Verification
- From the Virtual Machine Overview page, click the Disks tab.
- Check to ensure that the secret, config map, or service account that you removed is no longer included in the list of disks.
8.12. Installing VirtIO driver on an existing Windows virtual machine Copiar enlaceEnlace copiado en el portapapeles!
8.12.1. Understanding VirtIO drivers Copiar enlaceEnlace copiado en el portapapeles!
VirtIO drivers are paravirtualized device drivers required for Microsoft Windows virtual machines to run in OpenShift Virtualization. The supported drivers are available in the
container-native-virtualization/virtio-win
The
container-native-virtualization/virtio-win
After the drivers are installed, the
container-native-virtualization/virtio-win
See also: Installing Virtio drivers on a new Windows virtual machine.
8.12.2. Supported VirtIO drivers for Microsoft Windows virtual machines Copiar enlaceEnlace copiado en el portapapeles!
| Driver name | Hardware ID | Description |
|---|---|---|
| viostor |
VEN_1AF4&DEV_1001 | The block driver. Sometimes displays as an SCSI Controller in the Other devices group. |
| viorng |
VEN_1AF4&DEV_1005 | The entropy source driver. Sometimes displays as a PCI Device in the Other devices group. |
| NetKVM |
VEN_1AF4&DEV_1000 | The network driver. Sometimes displays as an Ethernet Controller in the Other devices group. Available only if a VirtIO NIC is configured. |
8.12.3. Adding VirtIO drivers container disk to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization distributes VirtIO drivers for Microsoft Windows as a container disk, which is available from the Red Hat Ecosystem Catalog. To install these drivers to a Windows virtual machine, attach the
container-native-virtualization/virtio-win
Prerequisites
-
Download the container disk from the Red Hat Ecosystem Catalog. This is not mandatory, because the container disk will be downloaded from the Red Hat registry if it not already present in the cluster, but it can reduce installation time.
container-native-virtualization/virtio-win
Procedure
Add the
container disk as acontainer-native-virtualization/virtio-windisk in the Windows virtual machine configuration file. The container disk will be downloaded from the registry if it is not already present in the cluster.cdromspec: domain: devices: disks: - name: virtiocontainerdisk bootOrder: 21 cdrom: bus: sata volumes: - containerDisk: image: container-native-virtualization/virtio-win name: virtiocontainerdisk- 1
- OpenShift Virtualization boots virtual machine disks in the order defined in the
VirtualMachineconfiguration file. You can either define other disks for the virtual machine before thecontainer-native-virtualization/virtio-wincontainer disk or use the optionalbootOrderparameter to ensure the virtual machine boots from the correct disk. If you specify thebootOrderfor a disk, it must be specified for all disks in the configuration.
The disk is available once the virtual machine has started:
-
If you add the container disk to a running virtual machine, use in the CLI or reboot the virtual machine for the changes to take effect.
oc apply -f <vm.yaml> -
If the virtual machine is not running, use .
virtctl start <vm>
-
If you add the container disk to a running virtual machine, use
After the virtual machine has started, the VirtIO drivers can be installed from the attached SATA CD drive.
8.12.4. Installing VirtIO drivers on an existing Windows virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Install the VirtIO drivers from the attached SATA CD drive to an existing Windows virtual machine.
This procedure uses a generic approach to adding drivers to Windows. The process might differ slightly between versions of Windows. See the installation documentation for your version of Windows for specific installation steps.
Procedure
- Start the virtual machine and connect to a graphical console.
- Log in to a Windows user session.
Open Device Manager and expand Other devices to list any Unknown device.
-
Open the to identify the unknown device. Right-click the device and select Properties.
Device Properties - Click the Details tab and select Hardware Ids in the Property list.
- Compare the Value for the Hardware Ids with the supported VirtIO drivers.
-
Open the
- Right-click the device and select Update Driver Software.
- Click Browse my computer for driver software and browse to the attached SATA CD drive, where the VirtIO drivers are located. The drivers are arranged hierarchically according to their driver type, operating system, and CPU architecture.
- Click Next to install the driver.
- Repeat this process for all the necessary VirtIO drivers.
- After the driver installs, click Close to close the window.
- Reboot the virtual machine to complete the driver installation.
8.12.5. Removing the VirtIO container disk from a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
After installing all required VirtIO drivers to the virtual machine, the
container-native-virtualization/virtio-win
container-native-virtualization/virtio-win
Procedure
Edit the configuration file and remove the
and thedisk.volume$ oc edit vm <vm-name>spec: domain: devices: disks: - name: virtiocontainerdisk bootOrder: 2 cdrom: bus: sata volumes: - containerDisk: image: container-native-virtualization/virtio-win name: virtiocontainerdisk- Reboot the virtual machine for the changes to take effect.
8.13. Installing VirtIO driver on a new Windows virtual machine Copiar enlaceEnlace copiado en el portapapeles!
8.13.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Windows installation media accessible by the virtual machine, such as importing an ISO into a data volume and attaching it to the virtual machine.
8.13.2. Understanding VirtIO drivers Copiar enlaceEnlace copiado en el portapapeles!
VirtIO drivers are paravirtualized device drivers required for Microsoft Windows virtual machines to run in OpenShift Virtualization. The supported drivers are available in the
container-native-virtualization/virtio-win
The
container-native-virtualization/virtio-win
After the drivers are installed, the
container-native-virtualization/virtio-win
See also: Installing VirtIO driver on an existing Windows virtual machine.
8.13.3. Supported VirtIO drivers for Microsoft Windows virtual machines Copiar enlaceEnlace copiado en el portapapeles!
| Driver name | Hardware ID | Description |
|---|---|---|
| viostor |
VEN_1AF4&DEV_1001 | The block driver. Sometimes displays as an SCSI Controller in the Other devices group. |
| viorng |
VEN_1AF4&DEV_1005 | The entropy source driver. Sometimes displays as a PCI Device in the Other devices group. |
| NetKVM |
VEN_1AF4&DEV_1000 | The network driver. Sometimes displays as an Ethernet Controller in the Other devices group. Available only if a VirtIO NIC is configured. |
8.13.4. Adding VirtIO drivers container disk to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization distributes VirtIO drivers for Microsoft Windows as a container disk, which is available from the Red Hat Ecosystem Catalog. To install these drivers to a Windows virtual machine, attach the
container-native-virtualization/virtio-win
Prerequisites
-
Download the container disk from the Red Hat Ecosystem Catalog. This is not mandatory, because the container disk will be downloaded from the Red Hat registry if it not already present in the cluster, but it can reduce installation time.
container-native-virtualization/virtio-win
Procedure
Add the
container disk as acontainer-native-virtualization/virtio-windisk in the Windows virtual machine configuration file. The container disk will be downloaded from the registry if it is not already present in the cluster.cdromspec: domain: devices: disks: - name: virtiocontainerdisk bootOrder: 21 cdrom: bus: sata volumes: - containerDisk: image: container-native-virtualization/virtio-win name: virtiocontainerdisk- 1
- OpenShift Virtualization boots virtual machine disks in the order defined in the
VirtualMachineconfiguration file. You can either define other disks for the virtual machine before thecontainer-native-virtualization/virtio-wincontainer disk or use the optionalbootOrderparameter to ensure the virtual machine boots from the correct disk. If you specify thebootOrderfor a disk, it must be specified for all disks in the configuration.
The disk is available once the virtual machine has started:
-
If you add the container disk to a running virtual machine, use in the CLI or reboot the virtual machine for the changes to take effect.
oc apply -f <vm.yaml> -
If the virtual machine is not running, use .
virtctl start <vm>
-
If you add the container disk to a running virtual machine, use
After the virtual machine has started, the VirtIO drivers can be installed from the attached SATA CD drive.
8.13.5. Installing VirtIO drivers during Windows installation Copiar enlaceEnlace copiado en el portapapeles!
Install the VirtIO drivers from the attached SATA CD driver during Windows installation.
This procedure uses a generic approach to the Windows installation and the installation method might differ between versions of Windows. See the documentation for the version of Windows that you are installing.
Procedure
- Start the virtual machine and connect to a graphical console.
- Begin the Windows installation process.
- Select the Advanced installation.
-
The storage destination will not be recognized until the driver is loaded. Click .
Load driver - The drivers are attached as a SATA CD drive. Click OK and browse the CD drive for the storage driver to load. The drivers are arranged hierarchically according to their driver type, operating system, and CPU architecture.
- Repeat the previous two steps for all required drivers.
- Complete the Windows installation.
8.13.6. Removing the VirtIO container disk from a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
After installing all required VirtIO drivers to the virtual machine, the
container-native-virtualization/virtio-win
container-native-virtualization/virtio-win
Procedure
Edit the configuration file and remove the
and thedisk.volume$ oc edit vm <vm-name>spec: domain: devices: disks: - name: virtiocontainerdisk bootOrder: 2 cdrom: bus: sata volumes: - containerDisk: image: container-native-virtualization/virtio-win name: virtiocontainerdisk- Reboot the virtual machine for the changes to take effect.
8.14. Advanced virtual machine management Copiar enlaceEnlace copiado en el portapapeles!
8.14.1. Working with resource quotas for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
Create and manage resource quotas for virtual machines.
8.14.1.1. Setting resource quota limits for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
Resource quotas that only use requests automatically work with virtual machines (VMs). If your resource quota uses limits, you must manually set resource limits on VMs. Resource limits must be at least 100 MiB larger than resource requests.
Procedure
Set limits for a VM by editing the
manifest. For example:VirtualMachineapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: with-limits spec: running: false template: spec: domain: # ... resources: requests: memory: 128Mi limits: memory: 256Mi1 - 1
- This configuration is supported because the
limits.memoryvalue is at least100Milarger than therequests.memoryvalue.
-
Save the manifest.
VirtualMachine
8.14.2. Specifying nodes for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can place virtual machines (VMs) on specific nodes by using node placement rules.
8.14.2.1. About node placement for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
To ensure that virtual machines (VMs) run on appropriate nodes, you can configure node placement rules. You might want to do this if:
- You have several VMs. To ensure fault tolerance, you want them to run on different nodes.
- You have two chatty VMs. To avoid redundant inter-node routing, you want the VMs to run on the same node.
- Your VMs require specific hardware features that are not present on all available nodes.
- You have a pod that adds capabilities to a node, and you want to place a VM on that node so that it can use those capabilities.
Virtual machine placement relies on any existing node placement rules for workloads. If workloads are excluded from specific nodes on the component level, virtual machines cannot be placed on those nodes.
You can use the following rule types in the
spec
VirtualMachine
nodeSelector- Allows virtual machines to be scheduled on nodes that are labeled with the key-value pair or pairs that you specify in this field. The node must have labels that exactly match all listed pairs.
affinityEnables you to use more expressive syntax to set rules that match nodes with virtual machines. For example, you can specify that a rule is a preference, rather than a hard requirement, so that virtual machines are still scheduled if the rule is not satisfied. Pod affinity, pod anti-affinity, and node affinity are supported for virtual machine placement. Pod affinity works for virtual machines because the
workload type is based on theVirtualMachineobject.PodNoteAffinity rules only apply during scheduling. OpenShift Container Platform does not reschedule running workloads if the constraints are no longer met.
tolerations- Allows virtual machines to be scheduled on nodes that have matching taints. If a taint is applied to a node, that node only accepts virtual machines that tolerate the taint.
8.14.2.2. Node placement examples Copiar enlaceEnlace copiado en el portapapeles!
The following example YAML file snippets use
nodePlacement
affinity
tolerations
8.14.2.2.1. Example: VM node placement with nodeSelector Copiar enlaceEnlace copiado en el portapapeles!
In this example, the virtual machine requires a node that has metadata containing both
example-key-1 = example-value-1
example-key-2 = example-value-2
If there are no nodes that fit this description, the virtual machine is not scheduled.
Example VM manifest
metadata:
name: example-vm-node-selector
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
template:
spec:
nodeSelector:
example-key-1: example-value-1
example-key-2: example-value-2
...
8.14.2.2.2. Example: VM node placement with pod affinity and pod anti-affinity Copiar enlaceEnlace copiado en el portapapeles!
In this example, the VM must be scheduled on a node that has a running pod with the label
example-key-1 = example-value-1
If possible, the VM is not scheduled on a node that has any pod with the label
example-key-2 = example-value-2
Example VM manifest
metadata:
name: example-vm-pod-affinity
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: example-key-1
operator: In
values:
- example-value-1
topologyKey: kubernetes.io/hostname
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: example-key-2
operator: In
values:
- example-value-2
topologyKey: kubernetes.io/hostname
...
- 1
- If you use the
requiredDuringSchedulingIgnoredDuringExecutionrule type, the VM is not scheduled if the constraint is not met. - 2
- If you use the
preferredDuringSchedulingIgnoredDuringExecutionrule type, the VM is still scheduled if the constraint is not met, as long as all required constraints are met.
8.14.2.2.3. Example: VM node placement with node affinity Copiar enlaceEnlace copiado en el portapapeles!
In this example, the VM must be scheduled on a node that has the label
example.io/example-key = example-value-1
example.io/example-key = example-value-2
If possible, the scheduler avoids nodes that have the label
example-node-label-key = example-node-label-value
Example VM manifest
metadata:
name: example-vm-node-affinity
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: example.io/example-key
operator: In
values:
- example-value-1
- example-value-2
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: example-node-label-key
operator: In
values:
- example-node-label-value
...
- 1
- If you use the
requiredDuringSchedulingIgnoredDuringExecutionrule type, the VM is not scheduled if the constraint is not met. - 2
- If you use the
preferredDuringSchedulingIgnoredDuringExecutionrule type, the VM is still scheduled if the constraint is not met, as long as all required constraints are met.
8.14.2.2.4. Example: VM node placement with tolerations Copiar enlaceEnlace copiado en el portapapeles!
In this example, nodes that are reserved for virtual machines are already labeled with the
key=virtualization:NoSchedule
tolerations
A virtual machine that tolerates a taint is not required to schedule onto a node with that taint.
Example VM manifest
metadata:
name: example-vm-tolerations
apiVersion: kubevirt.io/v1
kind: VirtualMachine
spec:
tolerations:
- key: "key"
operator: "Equal"
value: "virtualization"
effect: "NoSchedule"
...
8.14.3. Configuring certificate rotation Copiar enlaceEnlace copiado en el portapapeles!
Configure certificate rotation parameters to replace existing certificates.
8.14.3.1. Configuring certificate rotation Copiar enlaceEnlace copiado en el portapapeles!
You can do this during OpenShift Virtualization installation in the web console or after installation in the
HyperConverged
Procedure
Open the
CR by running the following command:HyperConverged$ oc edit hco -n openshift-cnv kubevirt-hyperconvergedEdit the
fields as shown in the following example. To avoid overloading the system, ensure that all values are greater than or equal to 10 minutes. Express all values as strings that comply with the golangspec.certConfigParseDurationformat.apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: certConfig: ca: duration: 48h0m0s renewBefore: 24h0m0s1 server: duration: 24h0m0s2 renewBefore: 12h0m0s3 - Apply the YAML file to your cluster.
8.14.3.2. Troubleshooting certificate rotation parameters Copiar enlaceEnlace copiado en el portapapeles!
Deleting one or more
certConfig
-
The value of must be less than or equal to the value of
ca.renewBefore.ca.duration -
The value of must be less than or equal to the value of
server.duration.ca.duration -
The value of must be less than or equal to the value of
server.renewBefore.server.duration
If the default values conflict with these conditions, you will receive an error.
If you remove the
server.duration
24h0m0s
ca.duration
Example
certConfig:
ca:
duration: 4h0m0s
renewBefore: 1h0m0s
server:
duration: 4h0m0s
renewBefore: 4h0m0s
This results in the following error message:
error: hyperconvergeds.hco.kubevirt.io "kubevirt-hyperconverged" could not be patched: admission webhook "validate-hco.kubevirt.io" denied the request: spec.certConfig: ca.duration is smaller than server.duration
The error message only mentions the first conflict. Review all certConfig values before you proceed.
8.14.4. Automating management tasks Copiar enlaceEnlace copiado en el portapapeles!
You can automate OpenShift Virtualization management tasks by using Red Hat Ansible Automation Platform. Learn the basics by using an Ansible Playbook to create a new virtual machine.
8.14.4.1. About Red Hat Ansible Automation Copiar enlaceEnlace copiado en el portapapeles!
Ansible is an automation tool used to configure systems, deploy software, and perform rolling updates. Ansible includes support for OpenShift Virtualization, and Ansible modules enable you to automate cluster management tasks such as template, persistent volume claim, and virtual machine operations.
Ansible provides a way to automate OpenShift Virtualization management, which you can also accomplish by using the
oc
8.14.4.2. Automating virtual machine creation Copiar enlaceEnlace copiado en el portapapeles!
You can use the
kubevirt_vm
Prerequisites
- Red Hat Ansible Engine version 2.8 or newer
Procedure
Edit an Ansible Playbook YAML file so that it includes the
task:kubevirt_vmkubevirt_vm: namespace: name: cpu_cores: memory: disks: - name: volume: containerDisk: image: disk: bus:NoteThis snippet only includes the
portion of the playbook.kubevirt_vmEdit the values to reflect the virtual machine you want to create, including the
, the number ofnamespace, thecpu_cores, and thememory. For example:diskskubevirt_vm: namespace: default name: vm1 cpu_cores: 1 memory: 64Mi disks: - name: containerdisk volume: containerDisk: image: kubevirt/cirros-container-disk-demo:latest disk: bus: virtioIf you want the virtual machine to boot immediately after creation, add
to the YAML file. For example:state: runningkubevirt_vm: namespace: default name: vm1 state: running1 cpu_cores: 1- 1
- Changing this value to
state: absentdeletes the virtual machine, if it already exists.
Run the
command, using your playbook’s file name as the only argument:ansible-playbook$ ansible-playbook create-vm.yamlReview the output to determine if the play was successful:
Example output
(...) TASK [Create my first VM] ************************************************************************ changed: [localhost] PLAY RECAP ******************************************************************************************************** localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0If you did not include
in your playbook file and you want to boot the VM now, edit the file so that it includesstate: runningand run the playbook again:state: running$ ansible-playbook create-vm.yaml
To verify that the virtual machine was created, try to access the VM console.
8.14.4.3. Example: Ansible Playbook for creating virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can use the
kubevirt_vm
The following YAML file is an example of the
kubevirt_vm
---
- name: Ansible Playbook 1
hosts: localhost
connection: local
tasks:
- name: Create my first VM
kubevirt_vm:
namespace: default
name: vm1
cpu_cores: 1
memory: 64Mi
disks:
- name: containerdisk
volume:
containerDisk:
image: kubevirt/cirros-container-disk-demo:latest
disk:
bus: virtio
Additional information
8.14.5. Using EFI mode for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can boot a virtual machine (VM) in Extensible Firmware Interface (EFI) mode.
8.14.5.1. About EFI mode for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
Extensible Firmware Interface (EFI), like legacy BIOS, initializes hardware components and operating system image files when a computer starts. EFI supports more modern features and customization options than BIOS, enabling faster boot times.
It stores all the information about initialization and startup in a file with a
.efi
OpenShift Virtualization only supports a virtual machine (VM) with Secure Boot when using EFI mode. If Secure Boot is not enabled, the VM crashes repeatedly. However, the VM might not support Secure Boot. Before you boot a VM, verify that it supports Secure Boot by checking the VM settings.
8.14.5.2. Booting virtual machines in EFI mode Copiar enlaceEnlace copiado en el portapapeles!
You can configure a virtual machine to boot in EFI mode by editing the VM or VMI manifest.
Prerequisites
-
Install the OpenShift CLI ().
oc
Procedure
Create a YAML file that defines a VM object or a Virtual Machine Instance (VMI) object. Use the firmware stanza of the example YAML file:
Booting in EFI mode with secure boot active
apiversion: kubevirt.io/v1 kind: VirtualMachineInstance metadata: labels: special: vmi-secureboot name: vmi-secureboot spec: domain: devices: disks: - disk: bus: virtio name: containerdisk features: acpi: {} smm: enabled: true1 firmware: bootloader: efi: secureBoot: true2 ...- 1
- OpenShift Virtualization requires System Management Mode (
SMM) to be enabled for Secure Boot in EFI mode to occur. - 2
- OpenShift Virtualization only supports a virtual machine (VM) with Secure Boot when using EFI mode. If Secure Boot is not enabled, the VM crashes repeatedly. However, the VM might not support Secure Boot. Before you boot a VM, verify that it supports Secure Boot by checking the VM settings.
Apply the manifest to your cluster by running the following command:
$ oc create -f <file_name>.yaml
8.14.6. Configuring PXE booting for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
PXE booting, or network booting, is available in OpenShift Virtualization. Network booting allows a computer to boot and load an operating system or other program without requiring a locally attached storage device. For example, you can use it to choose your desired OS image from a PXE server when deploying a new host.
8.14.6.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- A Linux bridge must be connected.
- The PXE server must be connected to the same VLAN as the bridge.
8.14.6.2. PXE booting with a specified MAC address Copiar enlaceEnlace copiado en el portapapeles!
As an administrator, you can boot a client over the network by first creating a
NetworkAttachmentDefinition
Prerequisites
- A Linux bridge must be connected.
- The PXE server must be connected to the same VLAN as the bridge.
Procedure
Configure a PXE network on the cluster:
Create the network attachment definition file for PXE network
:pxe-net-confapiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: pxe-net-conf spec: config: '{ "cniVersion": "0.3.1", "name": "pxe-net-conf", "plugins": [ { "type": "cnv-bridge", "bridge": "br1", "vlan": 11 }, { "type": "cnv-tuning"2 } ] }'NoteThe virtual machine instance will be attached to the bridge
through an access port with the requested VLAN.br1
Create the network attachment definition by using the file you created in the previous step:
$ oc create -f pxe-net-conf.yamlEdit the virtual machine instance configuration file to include the details of the interface and network.
Specify the network and MAC address, if required by the PXE server. If the MAC address is not specified, a value is assigned automatically.
Ensure that
is set tobootOrderso that the interface boots first. In this example, the interface is connected to a network called1:<pxe-net>interfaces: - masquerade: {} name: default - bridge: {} name: pxe-net macAddress: de:00:00:00:00:de bootOrder: 1NoteBoot order is global for interfaces and disks.
Assign a boot device number to the disk to ensure proper booting after operating system provisioning.
Set the disk
value tobootOrder:2devices: disks: - disk: bus: virtio name: containerdisk bootOrder: 2Specify that the network is connected to the previously created network attachment definition. In this scenario,
is connected to the network attachment definition called<pxe-net>:<pxe-net-conf>networks: - name: default pod: {} - name: pxe-net multus: networkName: pxe-net-conf
Create the virtual machine instance:
$ oc create -f vmi-pxe-boot.yaml
Example output
virtualmachineinstance.kubevirt.io "vmi-pxe-boot" created
Wait for the virtual machine instance to run:
$ oc get vmi vmi-pxe-boot -o yaml | grep -i phase phase: RunningView the virtual machine instance using VNC:
$ virtctl vnc vmi-pxe-boot- Watch the boot screen to verify that the PXE boot is successful.
Log in to the virtual machine instance:
$ virtctl console vmi-pxe-bootVerify the interfaces and MAC address on the virtual machine and that the interface connected to the bridge has the specified MAC address. In this case, we used
for the PXE boot, without an IP address. The other interface,eth1, got an IP address from OpenShift Container Platform.eth0$ ip addr
Example output
...
3. eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether de:00:00:00:00:de brd ff:ff:ff:ff:ff:ff
8.14.6.3. Template: Virtual machine instance configuration file for PXE booting Copiar enlaceEnlace copiado en el portapapeles!
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:
creationTimestamp: null
labels:
special: vmi-pxe-boot
name: vmi-pxe-boot
spec:
domain:
devices:
disks:
- disk:
bus: virtio
name: containerdisk
bootOrder: 2
- disk:
bus: virtio
name: cloudinitdisk
interfaces:
- masquerade: {}
name: default
- bridge: {}
name: pxe-net
macAddress: de:00:00:00:00:de
bootOrder: 1
machine:
type: ""
resources:
requests:
memory: 1024M
networks:
- name: default
pod: {}
- multus:
networkName: pxe-net-conf
name: pxe-net
terminationGracePeriodSeconds: 0
volumes:
- name: containerdisk
containerDisk:
image: kubevirt/fedora-cloud-container-disk-demo
- cloudInitNoCloud:
userData: |
#!/bin/bash
echo "fedora" | passwd fedora --stdin
name: cloudinitdisk
status: {}
8.14.7. Managing guest memory Copiar enlaceEnlace copiado en el portapapeles!
If you want to adjust guest memory settings to suit a specific use case, you can do so by editing the guest’s YAML configuration file. OpenShift Virtualization allows you to configure guest memory overcommitment and disable guest memory overhead accounting.
The following procedures increase the chance that virtual machine processes will be killed due to memory pressure. Proceed only if you understand the risks.
8.14.7.1. Configuring guest memory overcommitment Copiar enlaceEnlace copiado en el portapapeles!
If your virtual workload requires more memory than available, you can use memory overcommitment to allocate all or most of the host’s memory to your virtual machine instances (VMIs). Enabling memory overcommitment means that you can maximize resources that are normally reserved for the host.
For example, if the host has 32 GB RAM, you can use memory overcommitment to fit 8 virtual machines (VMs) with 4 GB RAM each. This allocation works under the assumption that the virtual machines will not use all of their memory at the same time.
Memory overcommitment increases the potential for virtual machine processes to be killed due to memory pressure (OOM killed).
The potential for a VM to be OOM killed varies based on your specific configuration, node memory, available swap space, virtual machine memory consumption, the use of kernel same-page merging (KSM), and other factors.
Procedure
To explicitly tell the virtual machine instance that it has more memory available than was requested from the cluster, edit the virtual machine configuration file and set
to a higher value thanspec.domain.memory.guest. This process is called memory overcommitment.spec.domain.resources.requests.memoryIn this example,
is requested from the cluster, but the virtual machine instance is told that it has1024Mavailable. As long as there is enough free memory available on the node, the virtual machine instance will consume up to 2048M.2048Mkind: VirtualMachine spec: template: domain: resources: requests: memory: 1024M memory: guest: 2048MNoteThe same eviction rules as those for pods apply to the virtual machine instance if the node is under memory pressure.
Create the virtual machine:
$ oc create -f <file_name>.yaml
8.14.7.2. Disabling guest memory overhead accounting Copiar enlaceEnlace copiado en el portapapeles!
A small amount of memory is requested by each virtual machine instance in addition to the amount that you request. This additional memory is used for the infrastructure that wraps each
VirtualMachineInstance
Though it is not usually advisable, it is possible to increase the virtual machine instance density on the node by disabling guest memory overhead accounting.
Disabling guest memory overhead accounting increases the potential for virtual machine processes to be killed due to memory pressure (OOM killed).
The potential for a VM to be OOM killed varies based on your specific configuration, node memory, available swap space, virtual machine memory consumption, the use of kernel same-page merging (KSM), and other factors.
Procedure
To disable guest memory overhead accounting, edit the YAML configuration file and set the
value toovercommitGuestOverhead. This parameter is disabled by default.truekind: VirtualMachine spec: template: domain: resources: overcommitGuestOverhead: true requests: memory: 1024MNoteIf
is enabled, it adds the guest overhead to memory limits, if present.overcommitGuestOverheadCreate the virtual machine:
$ oc create -f <file_name>.yaml
8.14.8. Using huge pages with virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can use huge pages as backing memory for virtual machines in your cluster.
8.14.8.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Nodes must have pre-allocated huge pages configured.
8.14.8.2. What huge pages do Copiar enlaceEnlace copiado en el portapapeles!
Memory is managed in blocks known as pages. On most systems, a page is 4Ki. 1Mi of memory is equal to 256 pages; 1Gi of memory is 256,000 pages, and so on. CPUs have a built-in memory management unit that manages a list of these pages in hardware. The Translation Lookaside Buffer (TLB) is a small hardware cache of virtual-to-physical page mappings. If the virtual address passed in a hardware instruction can be found in the TLB, the mapping can be determined quickly. If not, a TLB miss occurs, and the system falls back to slower, software-based address translation, resulting in performance issues. Since the size of the TLB is fixed, the only way to reduce the chance of a TLB miss is to increase the page size.
A huge page is a memory page that is larger than 4Ki. On x86_64 architectures, there are two common huge page sizes: 2Mi and 1Gi. Sizes vary on other architectures. To use huge pages, code must be written so that applications are aware of them. Transparent Huge Pages (THP) attempt to automate the management of huge pages without application knowledge, but they have limitations. In particular, they are limited to 2Mi page sizes. THP can lead to performance degradation on nodes with high memory utilization or fragmentation due to defragmenting efforts of THP, which can lock memory pages. For this reason, some applications may be designed to (or recommend) usage of pre-allocated huge pages instead of THP.
In OpenShift Virtualization, virtual machines can be configured to consume pre-allocated huge pages.
8.14.8.3. Configuring huge pages for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can configure virtual machines to use pre-allocated huge pages by including the
memory.hugepages.pageSize
resources.requests.memory
The memory request must be divisible by the page size. For example, you cannot request
500Mi
1Gi
The memory layouts of the host and the guest OS are unrelated. Huge pages requested in the virtual machine manifest apply to QEMU. Huge pages inside the guest can only be configured based on the amount of available memory of the virtual machine instance.
If you edit a running virtual machine, the virtual machine must be rebooted for the changes to take effect.
Prerequisites
- Nodes must have pre-allocated huge pages configured.
Procedure
In your virtual machine configuration, add the
andresources.requests.memoryparameters to thememory.hugepages.pageSize. The following configuration snippet is for a virtual machine that requests a total ofspec.domainmemory with a page size of4Gi:1Gikind: VirtualMachine ... spec: domain: resources: requests: memory: "4Gi"1 memory: hugepages: pageSize: "1Gi"2 ...Apply the virtual machine configuration:
$ oc apply -f <virtual_machine>.yaml
8.14.9. Enabling dedicated resources for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
To improve performance, you can dedicate node resources, such as CPU, to a virtual machine.
8.14.9.1. About dedicated resources Copiar enlaceEnlace copiado en el portapapeles!
When you enable dedicated resources for your virtual machine, your virtual machine’s workload is scheduled on CPUs that will not be used by other processes. By using dedicated resources, you can improve the performance of the virtual machine and the accuracy of latency predictions.
8.14.9.2. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
The CPU Manager must be configured on the node. Verify that the node has the label before scheduling virtual machine workloads.
cpumanager = true - The virtual machine must be powered off.
8.14.9.3. Enabling dedicated resources for a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can enable dedicated resources for a virtual machine in the Details tab. Virtual machines that were created by using a Red Hat template or the wizard can be enabled with dedicated resources.
Procedure
- Click Workloads → Virtual Machines from the side menu.
- Select a virtual machine to open the Virtual Machine tab.
- Click the Details tab.
- Click the pencil icon to the right of the Dedicated Resources field to open the Dedicated Resources window.
- Select Schedule this workload with dedicated resources (guaranteed policy).
- Click Save.
8.14.10. Scheduling virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can schedule a virtual machine (VM) on a node by ensuring that the VM’s CPU model and policy attribute are matched for compatibility with the CPU models and policy attributes supported by the node.
8.14.10.1. Understanding policy attributes Copiar enlaceEnlace copiado en el portapapeles!
You can schedule a virtual machine (VM) by specifying a policy attribute and a CPU feature that is matched for compatibility when the VM is scheduled on a node. A policy attribute specified for a VM determines how that VM is scheduled on a node.
| Policy attribute | Description |
|---|---|
| force | The VM is forced to be scheduled on a node. This is true even if the host CPU does not support the VM’s CPU. |
| require | Default policy that applies to a VM if the VM is not configured with a specific CPU model and feature specification. If a node is not configured to support CPU node discovery with this default policy attribute or any one of the other policy attributes, VMs are not scheduled on that node. Either the host CPU must support the VM’s CPU or the hypervisor must be able to emulate the supported CPU model. |
| optional | The VM is added to a node if that VM is supported by the host’s physical machine CPU. |
| disable | The VM cannot be scheduled with CPU node discovery. |
| forbid | The VM is not scheduled even if the feature is supported by the host CPU and CPU node discovery is enabled. |
8.14.10.2. Setting a policy attribute and CPU feature Copiar enlaceEnlace copiado en el portapapeles!
You can set a policy attribute and CPU feature for each virtual machine (VM) to ensure that it is scheduled on a node according to policy and feature. The CPU feature that you set is verified to ensure that it is supported by the host CPU or emulated by the hypervisor.
Procedure
Edit the
spec of your VM configuration file. The following example sets the CPU feature and thedomainpolicy for a virtual machine instance (VMI):requireapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: myvmi spec: domain: cpu: features: - name: apic1 policy: require2
8.14.10.3. Scheduling virtual machines with the supported CPU model Copiar enlaceEnlace copiado en el portapapeles!
You can configure a CPU model for a virtual machine (VM) or a virtual machine instance (VMI) to schedule it on a node where its CPU model is supported.
Procedure
Edit the
spec of your virtual machine configuration file. The following example shows a specific CPU model defined for a VMI:domainapiVersion: kubevirt.io/v1 kind: VirtualMachineInstance metadata: name: myvmi spec: domain: cpu: model: Conroe1 - 1
- CPU model for the VMI.
8.14.10.4. Scheduling virtual machines with the host model Copiar enlaceEnlace copiado en el portapapeles!
When the CPU model for a virtual machine (VM) is set to
host-model
Procedure
Edit the
spec of your VM configuration file. The following example showsdomainbeing specified for the virtual machine instance (VMI):host-modelapiVersion: kubevirt/v1alpha3 kind: VirtualMachineInstance metadata: name: myvmi spec: domain: cpu: model: host-model1 - 1
- The VM or VMI that inherits the CPU model of the node where it is scheduled.
8.14.11. Configuring PCI passthrough Copiar enlaceEnlace copiado en el portapapeles!
The Peripheral Component Interconnect (PCI) passthrough feature enables you to access and manage hardware devices from a virtual machine. When PCI passthrough is configured, the PCI devices function as if they were physically attached to the guest operating system.
Cluster administrators can expose and manage host devices that are permitted to be used in the cluster by using the
oc
8.14.11.1. About preparing a host device for PCI passthrough Copiar enlaceEnlace copiado en el portapapeles!
To prepare a host device for PCI passthrough by using the CLI, create a
MachineConfig
permittedHostDevices
HyperConverged
permittedHostDevices
To remove a PCI host device from the cluster by using the CLI, delete the PCI device information from the
HyperConverged
8.14.11.1.1. Adding kernel arguments to enable the IOMMU driver Copiar enlaceEnlace copiado en el portapapeles!
To enable the IOMMU (Input-Output Memory Management Unit) driver in the kernel, create the
MachineConfig
Prerequisites
- Administrative privilege to a working OpenShift Container Platform cluster.
- Intel or AMD CPU hardware.
- Intel Virtualization Technology for Directed I/O extensions or AMD IOMMU in the BIOS (Basic Input/Output System) is enabled.
Procedure
Create a
object that identifies the kernel argument. The following example shows a kernel argument for an Intel CPU.MachineConfigapiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker1 name: 100-worker-iommu2 spec: config: ignition: version: 3.2.0 kernelArguments: - intel_iommu=on3 ...Create the new
object:MachineConfig$ oc create -f 100-worker-kernel-arg-iommu.yaml
Verification
Verify that the new
object was added.MachineConfig$ oc get MachineConfig
8.14.11.1.2. Binding PCI devices to the VFIO driver Copiar enlaceEnlace copiado en el portapapeles!
To bind PCI devices to the VFIO (Virtual Function I/O) driver, obtain the values for
vendor-ID
device-ID
MachineConfig
MachineConfig
/etc/modprobe.d/vfio.conf
Prerequisites
- You added kernel arguments to enable IOMMU for the CPU.
Procedure
Run the
command to obtain thelspciand thevendor-IDfor the PCI device.device-ID$ lspci -nnv | grep -i nvidiaExample output
02:01.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100 PCIe 32GB] [10de:1eb8] (rev a1)Create a Butane config file,
, binding the PCI device to the VFIO driver.100-worker-vfiopci.buNoteSee "Creating machine configs with Butane" for information about Butane.
Example
variant: openshift version: 4.8.0 metadata: name: 100-worker-vfiopci labels: machineconfiguration.openshift.io/role: worker1 storage: files: - path: /etc/modprobe.d/vfio.conf mode: 0644 overwrite: true contents: inline: | options vfio-pci ids=10de:1eb82 - path: /etc/modules-load.d/vfio-pci.conf3 mode: 0644 overwrite: true contents: inline: vfio-pci- 1
- Applies the new kernel argument only to worker nodes.
- 2
- Specify the previously determined
vendor-IDvalue (10de) and thedevice-IDvalue (1eb8) to bind a single device to the VFIO driver. You can add a list of multiple devices with their vendor and device information. - 3
- The file that loads the vfio-pci kernel module on the worker nodes.
Use Butane to generate a
object file,MachineConfig, containing the configuration to be delivered to the worker nodes:100-worker-vfiopci.yaml$ butane 100-worker-vfiopci.bu -o 100-worker-vfiopci.yamlApply the
object to the worker nodes:MachineConfig$ oc apply -f 100-worker-vfiopci.yamlVerify that the
object was added.MachineConfig$ oc get MachineConfigExample output
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 00-master d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 00-worker d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 01-master-container-runtime d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 01-master-kubelet d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 01-worker-container-runtime d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 01-worker-kubelet d3da910bfa9f4b599af4ed7f5ac270d55950a3a1 3.2.0 25h 100-worker-iommu 3.2.0 30s 100-worker-vfiopci-configuration 3.2.0 30s
Verification
Verify that the VFIO driver is loaded.
$ lspci -nnk -d 10de:The output confirms that the VFIO driver is being used.
Example output
04:00.0 3D controller [0302]: NVIDIA Corporation GP102GL [Tesla P40] [10de:1eb8] (rev a1) Subsystem: NVIDIA Corporation Device [10de:1eb8] Kernel driver in use: vfio-pci Kernel modules: nouveau
8.14.11.1.3. Exposing PCI host devices in the cluster using the CLI Copiar enlaceEnlace copiado en el portapapeles!
To expose PCI host devices in the cluster, add details about the PCI devices to the
spec.permittedHostDevices.pciHostDevices
HyperConverged
Procedure
Edit the
CR in your default editor by running the following command:HyperConverged$ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnvAdd the PCI device information to the
array. For example:spec.permittedHostDevices.pciHostDevicesExample configuration file
apiVersion: hco.kubevirt.io/v1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: permittedHostDevices:1 pciHostDevices:2 - pciDeviceSelector: "10DE:1DB6"3 resourceName: "nvidia.com/GV100GL_Tesla_V100"4 - pciDeviceSelector: "10DE:1EB8" resourceName: "nvidia.com/TU104GL_Tesla_T4" - pciDeviceSelector: "8086:6F54" resourceName: "intel.com/qat" externalResourceProvider: true5 ...- 1
- The host devices that are permitted to be used in the cluster.
- 2
- The list of PCI devices available on the node.
- 3
- The
vendor-IDand thedevice-IDrequired to identify the PCI device. - 4
- The name of a PCI host device.
- 5
- Optional: Setting this field to
trueindicates that the resource is provided by an external device plugin. OpenShift Virtualization allows the usage of this device in the cluster but leaves the allocation and monitoring to an external device plugin.
NoteThe above example snippet shows two PCI host devices that are named
andnvidia.com/GV100GL_Tesla_V100added to the list of permitted host devices in thenvidia.com/TU104GL_Tesla_T4CR. These devices have been tested and verified to work with OpenShift Virtualization.HyperConverged- Save your changes and exit the editor.
Verification
Verify that the PCI host devices were added to the node by running the following command. The example output shows that there is one device each associated with the
,nvidia.com/GV100GL_Tesla_V100, andnvidia.com/TU104GL_Tesla_T4resource names.intel.com/qat$ oc describe node <node_name>Example output
Capacity: cpu: 64 devices.kubevirt.io/kvm: 110 devices.kubevirt.io/tun: 110 devices.kubevirt.io/vhost-net: 110 ephemeral-storage: 915128Mi hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 131395264Ki nvidia.com/GV100GL_Tesla_V100 1 nvidia.com/TU104GL_Tesla_T4 1 intel.com/qat: 1 pods: 250 Allocatable: cpu: 63500m devices.kubevirt.io/kvm: 110 devices.kubevirt.io/tun: 110 devices.kubevirt.io/vhost-net: 110 ephemeral-storage: 863623130526 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 130244288Ki nvidia.com/GV100GL_Tesla_V100 1 nvidia.com/TU104GL_Tesla_T4 1 intel.com/qat: 1 pods: 250
8.14.11.1.4. Removing PCI host devices from the cluster using the CLI Copiar enlaceEnlace copiado en el portapapeles!
To remove a PCI host device from the cluster, delete the information for that device from the
HyperConverged
Procedure
Edit the
CR in your default editor by running the following command:HyperConverged$ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnvRemove the PCI device information from the
array by deleting thespec.permittedHostDevices.pciHostDevices,pciDeviceSelectorandresourceName(if applicable) fields for the appropriate device. In this example, theexternalResourceProviderresource has been deleted.intel.com/qatExample configuration file
apiVersion: hco.kubevirt.io/v1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: permittedHostDevices: pciHostDevices: - pciDeviceSelector: "10DE:1DB6" resourceName: "nvidia.com/GV100GL_Tesla_V100" - pciDeviceSelector: "10DE:1EB8" resourceName: "nvidia.com/TU104GL_Tesla_T4" ...- Save your changes and exit the editor.
Verification
Verify that the PCI host device was removed from the node by running the following command. The example output shows that there are zero devices associated with the
resource name.intel.com/qat$ oc describe node <node_name>Example output
Capacity: cpu: 64 devices.kubevirt.io/kvm: 110 devices.kubevirt.io/tun: 110 devices.kubevirt.io/vhost-net: 110 ephemeral-storage: 915128Mi hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 131395264Ki nvidia.com/GV100GL_Tesla_V100 1 nvidia.com/TU104GL_Tesla_T4 1 intel.com/qat: 0 pods: 250 Allocatable: cpu: 63500m devices.kubevirt.io/kvm: 110 devices.kubevirt.io/tun: 110 devices.kubevirt.io/vhost-net: 110 ephemeral-storage: 863623130526 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 130244288Ki nvidia.com/GV100GL_Tesla_V100 1 nvidia.com/TU104GL_Tesla_T4 1 intel.com/qat: 0 pods: 250
8.14.11.2. Configuring virtual machines for PCI passthrough Copiar enlaceEnlace copiado en el portapapeles!
After the PCI devices have been added to the cluster, you can assign them to virtual machines. The PCI devices are now available as if they are physically connected to the virtual machines.
8.14.11.2.1. Assigning a PCI device to a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
When a PCI device is available in a cluster, you can assign it to a virtual machine and enable PCI passthrough.
Procedure
Assign the PCI device to a virtual machine as a host device.
Example
apiVersion: kubevirt.io/v1 kind: VirtualMachine spec: domain: devices: hostDevices: - deviceName: nvidia.com/TU104GL_Tesla_T41 name: hostdevices1- 1
- The name of the PCI device that is permitted on the cluster as a host device. The virtual machine can access this host device.
Verification
Use the following command to verify that the host device is available from the virtual machine.
$ lspci -nnk | grep NVIDIAExample output
$ 02:01.0 3D controller [0302]: NVIDIA Corporation GV100GL [Tesla V100 PCIe 32GB] [10de:1eb8] (rev a1)
8.14.12. Configuring a watchdog Copiar enlaceEnlace copiado en el portapapeles!
Expose a watchdog by configuring the virtual machine (VM) for a watchdog device, installing the watchdog, and starting the watchdog service.
8.14.12.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
The virtual machine must have kernel support for an watchdog device. Red Hat Enterprise Linux (RHEL) images support
i6300esb.i6300esb
8.14.12.2. Defining a watchdog device Copiar enlaceEnlace copiado en el portapapeles!
Define how the watchdog proceeds when the operating system (OS) no longer responds.
Table 8.3. Available actions
|
| The virtual machine (VM) powers down immediately. If
|
|
| The VM reboots in place and the guest OS cannot react. Because the length of time required for the guest OS to reboot can cause liveness probes to timeout, use of this option is discouraged. This timeout can extend the time it takes the VM to reboot if cluster-level protections notice the liveness probe failed and forcibly reschedule it. |
|
| The VM gracefully powers down by stopping all services. |
Procedure
Create a YAML file with the following contents:
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: labels: kubevirt.io/vm: vm2-rhel84-watchdog name: <vm-name> spec: running: false template: metadata: labels: kubevirt.io/vm: vm2-rhel84-watchdog spec: domain: devices: watchdog: name: <watchdog> i6300esb: action: "poweroff"1 ...- 1
- Specify the
watchdogaction (poweroff,reset, orshutdown).
The example above configures the
watchdog device on a RHEL8 VM with the poweroff action and exposes the device asi6300esb./dev/watchdogThis device can now be used by the watchdog binary.
Apply the YAML file to your cluster by running the following command:
$ oc apply -f <file_name>.yaml
This procedure is provided for testing watchdog functionality only and must not be run on production machines.
Run the following command to verify that the VM is connected to the watchdog device:
$ lspci | grep watchdog -iRun one of the following commands to confirm the watchdog is active:
Trigger a kernel panic:
# echo c > /proc/sysrq-triggerTerminate the watchdog service:
# pkill -9 watchdog
8.14.12.3. Installing a watchdog device Copiar enlaceEnlace copiado en el portapapeles!
Install the
watchdog
Procedure
As a root user, install the
package and dependencies:watchdog# yum install watchdogUncomment the following line in the
file, and save the changes:/etc/watchdog.conf#watchdog-device = /dev/watchdogEnable the watchdog service to start on boot:
# systemctl enable --now watchdog.service
8.15. Importing virtual machines Copiar enlaceEnlace copiado en el portapapeles!
8.15.1. TLS certificates for data volume imports Copiar enlaceEnlace copiado en el portapapeles!
8.15.1.1. Adding TLS certificates for authenticating data volume imports Copiar enlaceEnlace copiado en el portapapeles!
TLS certificates for registry or HTTPS endpoints must be added to a config map to import data from these sources. This config map must be present in the namespace of the destination data volume.
Create the config map by referencing the relative file path for the TLS certificate.
Procedure
Ensure you are in the correct namespace. The config map can only be referenced by data volumes if it is in the same namespace.
$ oc get nsCreate the config map:
$ oc create configmap <configmap-name> --from-file=</path/to/file/ca.pem>
8.15.1.2. Example: Config map created from a TLS certificate Copiar enlaceEnlace copiado en el portapapeles!
The following example is of a config map created from
ca.pem
apiVersion: v1
kind: ConfigMap
metadata:
name: tls-certs
data:
ca.pem: |
-----BEGIN CERTIFICATE-----
... <base64 encoded cert> ...
-----END CERTIFICATE-----
8.15.2. Importing virtual machine images with data volumes Copiar enlaceEnlace copiado en el portapapeles!
Use the Containerized Data Importer (CDI) to import a virtual machine image into a persistent volume claim (PVC) by using a data volume. You can attach a data volume to a virtual machine for persistent storage.
The virtual machine image can be hosted at an HTTP or HTTPS endpoint, or built into a container disk and stored in a container registry.
When you import a disk image into a PVC, the disk image is expanded to use the full storage capacity that is requested in the PVC. To use this space, the disk partitions and file system(s) in the virtual machine might need to be expanded.
The resizing procedure varies based on the operating system installed on the virtual machine. See the operating system documentation for details.
8.15.2.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- If the endpoint requires a TLS certificate, the certificate must be included in a config map in the same namespace as the data volume and referenced in the data volume configuration.
To import a container disk:
- You might need to prepare a container disk from a virtual machine image and store it in your container registry before importing it.
-
If the container registry does not have TLS, you must add the registry to the
insecureRegistriesfield of theHyperConvergedcustom resource before you can import a container disk from it.
- You might need to define a storage class or prepare CDI scratch space for this operation to complete successfully.
8.15.2.2. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
CDI now uses the OpenShift Container Platform cluster-wide proxy configuration.
8.15.2.3. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.15.2.4. Importing a virtual machine image into storage by using a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can import a virtual machine image into storage by using a data volume.
The virtual machine image can be hosted at an HTTP or HTTPS endpoint or the image can be built into a container disk and stored in a container registry.
You specify the data source for the image in a
VirtualMachine
Prerequisites
To import a virtual machine image you must have the following:
-
A virtual machine disk image in RAW, ISO, or QCOW2 format, optionally compressed by using or
xz.gz - An HTTP or HTTPS endpoint where the image is hosted, along with any authentication credentials needed to access the data source.
-
A virtual machine disk image in RAW, ISO, or QCOW2 format, optionally compressed by using
- To import a container disk, you must have a virtual machine image built into a container disk and stored in a container registry, along with any authentication credentials needed to access the data source.
- If the virtual machine must communicate with servers that use self-signed certificates or certificates not signed by the system CA bundle, you must create a config map in the same namespace as the data volume.
Procedure
If your data source requires authentication, create a
manifest, specifying the data source credentials, and save it asSecret:endpoint-secret.yamlapiVersion: v1 kind: Secret metadata: name: endpoint-secret1 labels: app: containerized-data-importer type: Opaque data: accessKeyId: ""2 secretKey: ""3 Apply the
manifest:Secret$ oc apply -f endpoint-secret.yamlEdit the
manifest, specifying the data source for the virtual machine image you want to import, and save it asVirtualMachine:vm-fedora-datavolume.yamlapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-fedora-datavolume name: vm-fedora-datavolume1 spec: dataVolumeTemplates: - metadata: creationTimestamp: null name: fedora-dv2 spec: storage: resources: requests: storage: 10Gi storageClassName: local source: http:3 url: "https://mirror.arizona.edu/fedora/linux/releases/35/Cloud/x86_64/images/Fedora-Cloud-Base-35-1.2.x86_64.qcow2"4 secretRef: endpoint-secret5 certConfigMap: ""6 status: {} running: true template: metadata: creationTimestamp: null labels: kubevirt.io/vm: vm-fedora-datavolume spec: domain: devices: disks: - disk: bus: virtio name: datavolumedisk1 machine: type: "" resources: requests: memory: 1.5Gi terminationGracePeriodSeconds: 60 volumes: - dataVolume: name: fedora-dv name: datavolumedisk1 status: {}- 1
- Specify the name of the virtual machine.
- 2
- Specify the name of the data volume.
- 3
- Specify
httpfor an HTTP or HTTPS endpoint. Specifyregistryfor a container disk image imported from a registry. - 4
- The source of the virtual machine image you want to import. This example references a virtual machine image at an HTTPS endpoint. An example of a container registry endpoint is
url: "docker://kubevirt/fedora-cloud-container-disk-demo:latest". - 5
- Required if you created a
Secretfor the data source. - 6
- Optional: Specify a CA certificate config map.
Create the virtual machine:
$ oc create -f vm-fedora-datavolume.yamlNoteThe
command creates the data volume and the virtual machine. The CDI controller creates an underlying PVC with the correct annotation and the import process begins. When the import is complete, the data volume status changes tooc create. You can start the virtual machine.SucceededData volume provisioning happens in the background, so there is no need to monitor the process.
Verification
The importer pod downloads the virtual machine image or container disk from the specified URL and stores it on the provisioned PV. View the status of the importer pod by running the following command:
$ oc get podsMonitor the data volume until its status is
by running the following command:Succeeded$ oc describe dv fedora-dv1 - 1
- Specify the data volume name that you defined in the
VirtualMachinemanifest.
Verify that provisioning is complete and that the virtual machine has started by accessing its serial console:
$ virtctl console vm-fedora-datavolume
8.15.3. Importing virtual machine images into block storage with data volumes Copiar enlaceEnlace copiado en el portapapeles!
You can import an existing virtual machine image into your OpenShift Container Platform cluster. OpenShift Virtualization uses data volumes to automate the import of data and the creation of an underlying persistent volume claim (PVC).
When you import a disk image into a PVC, the disk image is expanded to use the full storage capacity that is requested in the PVC. To use this space, the disk partitions and file system(s) in the virtual machine might need to be expanded.
The resizing procedure varies based on the operating system that is installed on the virtual machine. See the operating system documentation for details.
8.15.3.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- If you require scratch space according to the CDI supported operations matrix, you must first define a storage class or prepare CDI scratch space for this operation to complete successfully.
8.15.3.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.15.3.3. About block persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
A block persistent volume (PV) is a PV that is backed by a raw block device. These volumes do not have a file system and can provide performance benefits for virtual machines by reducing overhead.
Raw block volumes are provisioned by specifying
volumeMode: Block
8.15.3.4. Creating a local block persistent volume Copiar enlaceEnlace copiado en el portapapeles!
Create a local block persistent volume (PV) on a node by populating a file and mounting it as a loop device. You can then reference this loop device in a PV manifest as a
Block
Procedure
-
Log in as to the node on which to create the local PV. This procedure uses
rootfor its examples.node01 Create a file and populate it with null characters so that it can be used as a block device. The following example creates a file
with a size of 2Gb (20 100Mb blocks):loop10$ dd if=/dev/zero of=<loop10> bs=100M count=20Mount the
file as a loop device.loop10$ losetup </dev/loop10>d3 <loop10>1 2 Create a
manifest that references the mounted loop device.PersistentVolumekind: PersistentVolume apiVersion: v1 metadata: name: <local-block-pv10> annotations: spec: local: path: </dev/loop10>1 capacity: storage: <2Gi> volumeMode: Block2 storageClassName: local3 accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - <node01>4 Create the block PV.
# oc create -f <local-block-pv10.yaml>1 - 1
- The file name of the persistent volume created in the previous step.
8.15.3.5. Importing a virtual machine image into block storage by using a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can import a virtual machine image into block storage by using a data volume. You reference the data volume in a
VirtualMachine
Prerequisites
-
A virtual machine disk image in RAW, ISO, or QCOW2 format, optionally compressed by using or
xz.gz - An HTTP or HTTPS endpoint where the image is hosted, along with any authentication credentials needed to access the data source.
Procedure
If your data source requires authentication, create a
manifest, specifying the data source credentials, and save it asSecret:endpoint-secret.yamlapiVersion: v1 kind: Secret metadata: name: endpoint-secret1 labels: app: containerized-data-importer type: Opaque data: accessKeyId: ""2 secretKey: ""3 Apply the
manifest:Secret$ oc apply -f endpoint-secret.yamlCreate a
manifest, specifying the data source for the virtual machine image andDataVolumeforBlock.storage.volumeModeapiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: import-pv-datavolume1 spec: storageClassName: local2 source: http: url: "https://mirror.arizona.edu/fedora/linux/releases/35/Cloud/x86_64/images/Fedora-Cloud-Base-35-1.2.x86_64.qcow2"3 secretRef: endpoint-secret4 storage: volumeMode: Block5 resources: requests: storage: 10Gi- 1
- Specify the name of the data volume.
- 2
- Optional: Set the storage class or omit it to accept the cluster default.
- 3
- Specify the HTTP or HTTPS URL of the image to import.
- 4
- Required if you created a
Secretfor the data source. - 5
- The volume mode and access mode are detected automatically for known storage provisioners. Otherwise, specify
Block.
Create the data volume to import the virtual machine image:
$ oc create -f import-pv-datavolume.yaml
You can reference this data volume in a
VirtualMachine
8.15.3.6. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
CDI now uses the OpenShift Container Platform cluster-wide proxy configuration.
8.15.4. Importing a single Red Hat Virtualization virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can import a single Red Hat Virtualization (RHV) virtual machine into OpenShift Virtualization by using the VM Import wizard or the CLI.
Importing a RHV VM is a deprecated feature. Deprecated functionality is still included in OpenShift Virtualization and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
For the most recent list of major functionality that has been deprecated or removed within OpenShift Virtualization, refer to the Deprecated and removed features section of the OpenShift Virtualization release notes.
This feature will be replaced by the Migration Toolkit for Virtualization.
8.15.4.1. OpenShift Virtualization storage feature matrix Copiar enlaceEnlace copiado en el portapapeles!
The following table describes the OpenShift Virtualization storage types that support VM import.
| RHV VM import | |
|---|---|
| OpenShift Container Storage: RBD block-mode volumes | Yes |
| OpenShift Virtualization hostpath provisioner | No |
| Other multi-node writable storage | Yes [1] |
| Other single-node writable storage | Yes [2] |
- PVCs must request a ReadWriteMany access mode.
- PVCs must request a ReadWriteOnce access mode.
8.15.4.2. Prerequisites for importing a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
Importing a virtual machine from Red Hat Virtualization (RHV) into OpenShift Virtualization has the following prerequisites:
- You must have admin user privileges.
Storage:
- The OpenShift Virtualization local and shared persistent storage classes must support VM import.
- If you are using Ceph RBD block-mode volumes and the available storage space is too small for the virtual disk, the import process bar stops at 75% for more than 20 minutes and the migration does not succeed. No error message is displayed in the web console. BZ#1910019
Networks:
- The RHV and OpenShift Virtualization networks must either have the same name or be mapped to each other.
-
The RHV VM network interface must be ,
e1000, orrtl8139.virtio
VM disks:
-
The disk interface must be ,
sata, orvirtio_scsi.virtio - The disk must not be configured as a direct LUN.
-
The disk status must not be or
illegal.locked -
The storage type must be .
image - SCSI reservation must be disabled.
-
must be disabled.
ScsiGenericIO
-
The disk interface must be
VM configuration:
- If the VM uses GPU resources, the nodes providing the GPUs must be configured.
- The VM must not be configured for vGPU resources.
-
The VM must not have snapshots with disks in an state.
illegal - The VM must not have been created with OpenShift Container Platform and subsequently added to RHV.
- The VM must not be configured for USB devices.
-
The watchdog model must not be .
diag288
8.15.4.3. Importing a virtual machine with the VM Import wizard Copiar enlaceEnlace copiado en el portapapeles!
You can import a single virtual machine with the VM Import wizard.
Procedure
- In the web console, click Workloads → Virtual Machines.
- Click Create Virtual Machine and select Import with Wizard.
- Select Red Hat Virtualization (RHV) from the Provider list.
Select Connect to New Instance or a saved RHV instance.
If you select Connect to New Instance, fill in the following fields:
-
API URL: For example,
https://<RHV_Manager_FQDN>/ovirt-engine/api CA certificate: Click Browse to upload the RHV Manager CA certificate or paste the CA certificate into the field.
View the CA certificate by running the following command:
$ openssl s_client -connect <RHV_Manager_FQDN>:443 -showcerts < /dev/nullThe CA certificate is the second certificate in the output.
-
Username: RHV Manager user name, for example,
ocpadmin@internal - Password: RHV Manager password
-
API URL: For example,
- If you select a saved RHV instance, the wizard connects to the RHV instance using the saved credentials.
Click Check and Save and wait for the connection to complete.
NoteThe connection details are stored in a secret. If you add a provider with an incorrect URL, user name, or password, click Workloads → Secrets and delete the provider secret.
- Select a cluster and a virtual machine.
- Click Next.
- In the Review screen, review your settings.
- Optional: You can select Start virtual machine on creation.
Click Edit to update the following settings:
- General → Name: The VM name is limited to 63 characters.
General → Description: Optional description of the VM.
Storage Class: Select NFS or ocs-storagecluster-ceph-rbd.
If you select ocs-storagecluster-ceph-rbd, you must set the Volume Mode of the disk to Block.
- Advanced → Volume Mode: Select Block.
- Advanced → Volume Mode: Select Block.
- Networking → Network: You can select a network from a list of available network attachment definition objects.
Click Import or Review and Import, if you have edited the import settings.
A Successfully created virtual machine message and a list of resources created for the virtual machine are displayed. The virtual machine appears in Workloads → Virtual Machines.
Virtual machine wizard fields
| Name | Parameter | Description |
|---|---|---|
| Name | The name can contain lowercase letters (
| |
| Description | Optional description field. | |
| Operating System | The operating system that is selected for the virtual machine in the template. You cannot edit this field when creating a virtual machine from a template. | |
| Boot Source | Import via URL (creates PVC) | Import content from an image available from an HTTP or HTTPS endpoint. Example: Obtaining a URL link from the web page with the operating system image. |
| Clone existing PVC (creates PVC) | Select an existent persistent volume claim available on the cluster and clone it. | |
| Import via Registry (creates PVC) | Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Example:
| |
| PXE (network boot - adds network interface) | Boot an operating system from a server on the network. Requires a PXE bootable network attachment definition. | |
| Persistent Volume Claim project | Project name that you want to use for cloning the PVC. | |
| Persistent Volume Claim name | PVC name that should apply to this virtual machine template if you are cloning an existing PVC. | |
| Mount this as a CD-ROM boot source | A CD-ROM requires an additional disk for installing the operating system. Select the checkbox to add a disk and customize it later. | |
| Flavor | Tiny, Small, Medium, Large, Custom | Presets the amount of CPU and memory in a virtual machine template with predefined values that are allocated to the virtual machine, depending on the operating system associated with that template. If you choose a default template, you can override the
|
| Workload Type Note If you choose the incorrect Workload Type, there could be performance or resource utilization issues (such as a slow UI). | Desktop | A virtual machine configuration for use on a desktop. Ideal for consumption on a small scale. Recommended for use with the web console. |
| Server | Balances performance and it is compatible with a wide range of server workloads. | |
| High-Performance | A virtual machine configuration that is optimized for high-performance workloads. | |
| Start this virtual machine after creation. | This checkbox is selected by default and the virtual machine starts running after creation. Clear the checkbox if you do not want the virtual machine to start when it is created. |
Networking fields
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
Storage fields
| Name | Selection | Description |
|---|---|---|
| Source | Blank (creates PVC) | Create an empty disk. |
| Import via URL (creates PVC) | Import content via URL (HTTP or HTTPS endpoint). | |
| Use an existing PVC | Use a PVC that is already available in the cluster. | |
| Clone existing PVC (creates PVC) | Select an existing PVC available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Import content via container registry. | |
| Container (ephemeral) | Upload content from a container located in a registry accessible from the cluster. The container disk should be used only for read-only filesystems such as CD-ROMs or temporary virtual machines. | |
| Name | Name of the disk. The name can contain lowercase letters (
| |
| Size | Size of the disk in GiB. | |
| Type | Type of disk. Example: Disk or CD-ROM | |
| Interface | Type of disk device. Supported interfaces are virtIO, SATA, and SCSI. | |
| Storage Class | The storage class that is used to create the disk. | |
| Advanced → Volume Mode | Defines whether the persistent volume uses a formatted file system or raw block state. Default is Filesystem. |
Advanced storage settings
| Name | Parameter | Description |
|---|---|---|
| Volume Mode | Filesystem | Stores the virtual disk on a file system-based volume. |
| Block | Stores the virtual disk directly on the block volume. Only use
| |
| Access Mode [1] | Single User (RWO) | The disk can be mounted as read/write by a single node. |
| Shared Access (RWX) | The disk can be mounted as read/write by many nodes. | |
| Read Only (ROX) | The disk can be mounted as read-only by many nodes. |
- You can change the access mode by using the command line interface.
8.15.4.4. Importing a virtual machine with the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can import a virtual machine with the CLI by creating the
Secret
VirtualMachineImport
Secret
VirtualMachineImport
Optional: You can create a
ResourceMapping
VirtualMachineImport
ResourceMapping
The default target storage class must be NFS. Cinder does not support RHV VM import.
Procedure
Create the
CR by running the following command:Secret$ cat <<EOF | oc create -f - apiVersion: v1 kind: Secret metadata: name: rhv-credentials namespace: default1 type: Opaque stringData: ovirt: | apiUrl: <api_endpoint>2 username: ocpadmin@internal password:3 caCert: | -----BEGIN CERTIFICATE-----4 -----END CERTIFICATE----- EOF- 1
- Optional. You can specify a different namespace in all the CRs.
- 2
- Specify the API endpoint of the RHV Manager, for example,
\"https://www.example.com:8443/ovirt-engine/api" - 3
- Specify the password for
ocpadmin@internal. - 4
- Specify the RHV Manager CA certificate. You can obtain the CA certificate by running the following command:
$ openssl s_client -connect :443 -showcerts < /dev/nullOptional: Create a
CR if you want to separate the resource mapping from theResourceMappingCR by running the following command:VirtualMachineImport$ cat <<EOF | kubectl create -f - apiVersion: v2v.kubevirt.io/v1alpha1 kind: ResourceMapping metadata: name: resourcemapping_example namespace: default spec: ovirt: networkMappings: - source: name: <rhv_logical_network>/<vnic_profile>1 target: name: <target_network>2 type: pod storageMappings:3 - source: name: <rhv_storage_domain>4 target: name: <target_storage_class>5 volumeMode: <volume_mode>6 EOF- 1
- Specify the RHV logical network and vNIC profile.
- 2
- Specify the OpenShift Virtualization network.
- 3
- If storage mappings are specified in both the
ResourceMappingand theVirtualMachineImportCRs, theVirtualMachineImportCR takes precedence. - 4
- Specify the RHV storage domain.
- 5
- Specify
nfsorocs-storagecluster-ceph-rbd. - 6
- If you specified the
ocs-storagecluster-ceph-rbdstorage class, you must specifyBlockas the volume mode.
Create the
CR by running the following command:VirtualMachineImport$ cat <<EOF | oc create -f - apiVersion: v2v.kubevirt.io/v1beta1 kind: VirtualMachineImport metadata: name: vm-import namespace: default spec: providerCredentialsSecret: name: rhv-credentials namespace: default # resourceMapping:1 # name: resourcemapping-example # namespace: default targetVmName: vm_example2 startVm: true source: ovirt: vm: id: <source_vm_id>3 name: <source_vm_name>4 cluster: name: <source_cluster_name>5 mappings:6 networkMappings: - source: name: <source_logical_network>/<vnic_profile>7 target: name: <target_network>8 type: pod storageMappings:9 - source: name: <source_storage_domain>10 target: name: <target_storage_class>11 accessMode: <volume_access_mode>12 diskMappings: - source: id: <source_vm_disk_id>13 target: name: <target_storage_class>14 EOF- 1
- If you create a
ResourceMappingCR, uncomment theresourceMappingsection. - 2
- Specify the target VM name.
- 3
- Specify the source VM ID, for example,
80554327-0569-496b-bdeb-fcbbf52b827b. You can obtain the VM ID by enteringhttps://www.example.com/ovirt-engine/api/vms/in a web browser on the Manager machine to list all VMs. Locate the VM you want to import and its corresponding VM ID. You do not need to specify a VM name or cluster name. - 4
- If you specify the source VM name, you must also specify the source cluster. Do not specify the source VM ID.
- 5
- If you specify the source cluster, you must also specify the source VM name. Do not specify the source VM ID.
- 6
- If you create a
ResourceMappingCR, comment out themappingssection. - 7
- Specify the logical network and vNIC profile of the source VM.
- 8
- Specify the OpenShift Virtualization network.
- 9
- If storage mappings are specified in both the
ResourceMappingand theVirtualMachineImportCRs, theVirtualMachineImportCR takes precedence. - 10
- Specify the source storage domain.
- 11
- Specify the target storage class.
- 12
- Specify
ReadWriteOnce,ReadWriteMany, orReadOnlyMany. If no access mode is specified, {virt} determines the correct volume access mode based on the Host → Migration mode setting of the RHV VM or on the virtual disk access mode:-
If the RHV VM migration mode is , the default access mode is
Allow manual and automatic migration.ReadWriteMany -
If the RHV virtual disk access mode is , the default access mode is
ReadOnly.ReadOnlyMany -
For all other settings, the default access mode is .
ReadWriteOnce
-
If the RHV VM migration mode is
- 13
- Specify the source VM disk ID, for example,
8181ecc1-5db8-4193-9c92-3ddab3be7b05. You can obtain the disk ID by enteringhttps://www.example.com/ovirt-engine/api/vms/vm23in a web browser on the Manager machine and reviewing the VM details. - 14
- Specify the target storage class.
Follow the progress of the virtual machine import to verify that the import was successful:
$ oc get vmimports vm-import -n defaultThe output indicating a successful import resembles the following example:
Example output
... status: conditions: - lastHeartbeatTime: "2020-07-22T08:58:52Z" lastTransitionTime: "2020-07-22T08:58:52Z" message: Validation completed successfully reason: ValidationCompleted status: "True" type: Valid - lastHeartbeatTime: "2020-07-22T08:58:52Z" lastTransitionTime: "2020-07-22T08:58:52Z" message: 'VM specifies IO Threads: 1, VM has NUMA tune mode specified: interleave' reason: MappingRulesVerificationReportedWarnings status: "True" type: MappingRulesVerified - lastHeartbeatTime: "2020-07-22T08:58:56Z" lastTransitionTime: "2020-07-22T08:58:52Z" message: Copying virtual machine disks reason: CopyingDisks status: "True" type: Processing dataVolumes: - name: fedora32-b870c429-11e0-4630-b3df-21da551a48c0 targetVmName: fedora32
8.15.4.4.1. Creating a config map for importing a VM Copiar enlaceEnlace copiado en el portapapeles!
You can create a config map to map the Red Hat Virtualization (RHV) virtual machine operating system to an OpenShift Virtualization template if you want to override the default
vm-import-controller
The default
vm-import-controller
| RHV VM operating system | OpenShift Virtualization template |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Procedure
In a web browser, identify the REST API name of the RHV VM operating system by navigating to
. The operating system name appears in thehttp://<RHV_Manager_FQDN>/ovirt-engine/api/vms/<VM_ID>section of the XML output, as shown in the following example:<os>... <os> ... <type>rhel_8x64</type> </os>View a list of the available OpenShift Virtualization templates:
$ oc get templates -n openshift --show-labels | tr ',' '\n' | grep os.template.kubevirt.io | sed -r 's#os.template.kubevirt.io/(.*)=.*#\1#g' | sort -uExample output
fedora31 fedora32 ... rhel8.1 rhel8.2 ...- If an OpenShift Virtualization template that matches the RHV VM operating system does not appear in the list of available templates, create a template with the OpenShift Virtualization web console.
Create a config map to map the RHV VM operating system to the OpenShift Virtualization template:
$ cat <<EOF | oc create -f - apiVersion: v1 kind: ConfigMap metadata: name: os-configmap namespace: default1 data: guestos2common: | "Red Hat Enterprise Linux Server": "rhel" "CentOS Linux": "centos" "Fedora": "fedora" "Ubuntu": "ubuntu" "openSUSE": "opensuse" osinfo2common: | "<rhv-operating-system>": "<vm-template>"2 EOFConfig map example
$ cat <<EOF | oc apply -f - apiVersion: v1 kind: ConfigMap metadata: name: os-configmap namespace: default data: osinfo2common: | "other_linux": "fedora31" EOFVerify that the custom config map was created:
$ oc get cm -n default os-configmap -o yamlPatch the
config map to apply the new config map:vm-import-controller-config$ oc patch configmap vm-import-controller-config -n openshift-cnv --patch '{ "data": { "osConfigMap.name": "os-configmap", "osConfigMap.namespace": "default"1 } }'- 1
- Update the namespace if you changed it in the config map.
Verify that the template appears in the OpenShift Virtualization web console:
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machine Templates tab and find the template in the list.
8.15.4.5. Troubleshooting a virtual machine import Copiar enlaceEnlace copiado en el portapapeles!
8.15.4.5.1. Logs Copiar enlaceEnlace copiado en el portapapeles!
You can check the VM Import Controller pod log for errors.
Procedure
View the VM Import Controller pod name by running the following command:
$ oc get pods -n <namespace> | grep import1 - 1
- Specify the namespace of your imported virtual machine.
Example output
vm-import-controller-f66f7d-zqkz7 1/1 Running 0 4h49mView the VM Import Controller pod log by running the following command:
$ oc logs <vm-import-controller-f66f7d-zqkz7> -f -n <namespace>1 - 1
- Specify the VM Import Controller pod name and the namespace.
8.15.4.5.2. Error messages Copiar enlaceEnlace copiado en el portapapeles!
The following error message might appear:
The following error message is displayed in the VM Import Controller pod log and the progress bar stops at 10% if the OpenShift Virtualization storage PV is not suitable:
Failed to bind volumes: provisioning failed for PVCYou must use a compatible storage class. The Cinder storage class is not supported.
8.15.4.5.3. Known issues Copiar enlaceEnlace copiado en el portapapeles!
- If you are using Ceph RBD block-mode volumes and the available storage space is too small for the virtual disk, the import process bar stops at 75% for more than 20 minutes and the migration does not succeed. No error message is displayed in the web console. BZ#1910019
8.15.5. Importing a single VMware virtual machine or template Copiar enlaceEnlace copiado en el portapapeles!
You can import a VMware vSphere 6.5, 6.7, or 7.0 VM or VM template into OpenShift Virtualization by using the VM Import wizard. If you import a VM template, OpenShift Virtualization creates a virtual machine based on the template.
Importing a VMware VM is a deprecated feature. Deprecated functionality is still included in OpenShift Virtualization and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
For the most recent list of major functionality that has been deprecated or removed within OpenShift Virtualization, refer to the Deprecated and removed features section of the OpenShift Virtualization release notes.
This feature will be replaced by the Migration Toolkit for Virtualization.
8.15.5.1. OpenShift Virtualization storage feature matrix Copiar enlaceEnlace copiado en el portapapeles!
The following table describes the OpenShift Virtualization storage types that support VM import.
| VMware VM import | |
|---|---|
| OpenShift Container Storage: RBD block-mode volumes | Yes |
| OpenShift Virtualization hostpath provisioner | Yes |
| Other multi-node writable storage | Yes [1] |
| Other single-node writable storage | Yes [2] |
- PVCs must request a ReadWriteMany access mode.
- PVCs must request a ReadWriteOnce access mode.
8.15.5.2. Preparing a VDDK image Copiar enlaceEnlace copiado en el portapapeles!
The import process uses the VMware Virtual Disk Development Kit (VDDK) to copy the VMware virtual disk.
You can download the VDDK SDK, create a VDDK image, upload the image to an image registry, and add it to the
spec.vddkInitImage
HyperConverged
You can configure either an internal OpenShift Container Platform image registry or a secure external image registry for the VDDK image. The registry must be accessible to your OpenShift Virtualization environment.
Storing the VDDK image in a public registry might violate the terms of the VMware license.
8.15.5.2.1. Configuring an internal image registry Copiar enlaceEnlace copiado en el portapapeles!
You can configure the internal OpenShift Container Platform image registry on bare metal by updating the Image Registry Operator configuration.
You can access the registry directly, from within the OpenShift Container Platform cluster, or externally, by exposing the registry with a route.
Changing the image registry’s management state
To start the image registry, you must change the Image Registry Operator configuration’s
managementState
Removed
Managed
Procedure
Change
Image Registry Operator configuration frommanagementStatetoRemoved. For example:Managed$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'
Configuring registry storage for bare metal and other manual installations
As a cluster administrator, following installation you must configure your registry to use storage.
Prerequisites
-
You have access to the cluster as a user with the role.
cluster-admin - You have a cluster that uses manually-provisioned Red Hat Enterprise Linux CoreOS (RHCOS) nodes, such as bare metal.
You have provisioned persistent storage for your cluster, such as Red Hat OpenShift Container Storage.
ImportantOpenShift Container Platform supports
access for image registry storage when you have only one replica.ReadWriteOnceaccess also requires that the registry uses theReadWriteOncerollout strategy. To deploy an image registry that supports high availability with two or more replicas,Recreateaccess is required.ReadWriteMany- Must have 100Gi capacity.
Procedure
To configure your registry to use storage, change the
in thespec.storage.pvcresource.configs.imageregistry/clusterNoteWhen using shared storage, review your security settings to prevent outside access.
Verify that you do not have a registry pod:
$ oc get pod -n openshift-image-registry -l docker-registry=defaultExample output
No resourses found in openshift-image-registry namespaceNoteIf you do have a registry pod in your output, you do not need to continue with this procedure.
Check the registry configuration:
$ oc edit configs.imageregistry.operator.openshift.ioExample output
storage: pvc: claim:Leave the
field blank to allow the automatic creation of anclaimPVC.image-registry-storageCheck the
status:clusteroperator$ oc get clusteroperator image-registryExample output
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE image-registry 4.7 True False False 6h50mEnsure that your registry is set to managed to enable building and pushing of images.
Run:
$ oc edit configs.imageregistry/clusterThen, change the line
managementState: Removedto
managementState: Managed
Accessing registry directly from the cluster
You can access the registry from inside the cluster.
Procedure
Access the registry from the cluster by using internal routes:
Access the node by getting the node’s name:
$ oc get nodes$ oc debug nodes/<node_name>To enable access to tools such as
andocon the node, change your root directory topodman:/hostsh-4.2# chroot /hostLog in to the container image registry by using your access token:
sh-4.2# oc login -u kubeadmin -p <password_from_install_log> https://api-int.<cluster_name>.<base_domain>:6443sh-4.2# podman login -u kubeadmin -p $(oc whoami -t) image-registry.openshift-image-registry.svc:5000You should see a message confirming login, such as:
Login Succeeded!NoteYou can pass any value for the user name; the token contains all necessary information. Passing a user name that contains colons will result in a login failure.
Since the Image Registry Operator creates the route, it will likely be similar to
.default-route-openshift-image-registry.<cluster_name>Perform
andpodman pulloperations against your registry:podman pushImportantYou can pull arbitrary images, but if you have the system:registry role added, you can only push images to the registry in your project.
In the following examples, use:
Expand Component Value <registry_ip>
172.30.124.220<port>
5000<project>
openshift<image>
image<tag>
omitted (defaults to
)latestPull an arbitrary image:
sh-4.2# podman pull name.io/imageTag the new image with the form
. The project name must appear in this pull specification for OpenShift Container Platform to correctly place and later access the image in the registry:<registry_ip>:<port>/<project>/<image>sh-4.2# podman tag name.io/image image-registry.openshift-image-registry.svc:5000/openshift/imageNoteYou must have the
role for the specified project, which allows the user to write or push an image. Otherwise, thesystem:image-builderin the next step will fail. To test, you can create a new project to push the image.podman pushPush the newly tagged image to your registry:
sh-4.2# podman push image-registry.openshift-image-registry.svc:5000/openshift/image
Exposing a secure registry manually
Instead of logging in to the OpenShift Container Platform registry from within the cluster, you can gain external access to it by exposing it with a route. This allows you to log in to the registry from outside the cluster using the route address, and to tag and push images to an existing project by using the route host.
Prerequisites:
The following prerequisites are automatically performed:
- Deploy the Registry Operator.
- Deploy the Ingress Operator.
Procedure
You can expose the route by using
DefaultRoute
configs.imageregistry.operator.openshift.io
To expose the registry using
DefaultRoute
Set
toDefaultRoute:True$ oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=mergeLog in with
:podman$ HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}')$ podman login -u kubeadmin -p $(oc whoami -t) --tls-verify=false $HOST1 - 1
--tls-verify=falseis needed if the cluster’s default certificate for routes is untrusted. You can set a custom, trusted certificate as the default certificate with the Ingress Operator.
To expose the registry using custom routes:
Create a secret with your route’s TLS keys:
$ oc create secret tls public-route-tls \ -n openshift-image-registry \ --cert=</path/to/tls.crt> \ --key=</path/to/tls.key>This step is optional. If you do not create a secret, the route uses the default TLS configuration from the Ingress Operator.
On the Registry Operator:
spec: routes: - name: public-routes hostname: myregistry.mycorp.organization secretName: public-route-tls ...NoteOnly set
if you are providing a custom TLS configuration for the registry’s route.secretName
8.15.5.2.2. Configuring an external image registry Copiar enlaceEnlace copiado en el portapapeles!
If you use an external image registry for the VDDK image, you can add the external image registry’s certificate authorities to the OpenShift Container Platform cluster.
Optionally, you can create a pull secret from your Docker credentials and add it to your service account.
Adding certificate authorities to the cluster
You can add certificate authorities (CA) to the cluster for use when pushing and pulling images with the following procedure.
Prerequisites
- You must have cluster administrator privileges.
-
You must have access to the public certificates of the registry, usually a file located in the
hostname/ca.crtdirectory./etc/docker/certs.d/
Procedure
Create a
in theConfigMapnamespace containing the trusted certificates for the registries that use self-signed certificates. For each CA file, ensure the key in theopenshift-configis the hostname of the registry in theConfigMapformat:hostname[..port]$ oc create configmap registry-cas -n openshift-config \ --from-file=myregistry.corp.com..5000=/etc/docker/certs.d/myregistry.corp.com:5000/ca.crt \ --from-file=otherregistry.com=/etc/docker/certs.d/otherregistry.com/ca.crtUpdate the cluster image configuration:
$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-cas"}}}' --type=merge
Allowing pods to reference images from other secured registries
The
.dockercfg
$HOME/.docker/config.json
To pull a secured container image that is not from OpenShift Container Platform’s internal registry, you must create a pull secret from your Docker credentials and add it to your service account.
Procedure
If you already have a
file for the secured registry, you can create a secret from that file by running:.dockercfg$ oc create secret generic <pull_secret_name> \ --from-file=.dockercfg=<path/to/.dockercfg> \ --type=kubernetes.io/dockercfgOr if you have a
file:$HOME/.docker/config.json$ oc create secret generic <pull_secret_name> \ --from-file=.dockerconfigjson=<path/to/.docker/config.json> \ --type=kubernetes.io/dockerconfigjsonIf you do not already have a Docker credentials file for the secured registry, you can create a secret by running:
$ oc create secret docker-registry <pull_secret_name> \ --docker-server=<registry_server> \ --docker-username=<user_name> \ --docker-password=<password> \ --docker-email=<email>To use a secret for pulling images for pods, you must add the secret to your service account. The name of the service account in this example should match the name of the service account the pod uses. The default service account is
:default$ oc secrets link default <pull_secret_name> --for=pull
8.15.5.2.3. Creating and using a VDDK image Copiar enlaceEnlace copiado en el portapapeles!
You can download the VMware Virtual Disk Development Kit (VDDK), build a VDDK image, and push the VDDK image to your image registry. You then add the VDDK image to the
spec.vddkInitImage
HyperConverged
Prerequisites
- You must have access to an OpenShift Container Platform internal image registry or a secure external registry.
Procedure
Create and navigate to a temporary directory:
$ mkdir /tmp/<dir_name> && cd /tmp/<dir_name>- In a browser, navigate to VMware code and click SDKs.
- Under Compute Virtualization, click Virtual Disk Development Kit (VDDK).
- Select the VDDK version that corresponds to your VMware vSphere version, for example, VDDK 7.0 for vSphere 7.0, click Download, and then save the VDDK archive in the temporary directory.
Extract the VDDK archive:
$ tar -xzf VMware-vix-disklib-<version>.x86_64.tar.gzCreate a
:Dockerfile$ cat > Dockerfile <<EOF FROM busybox:latest COPY vmware-vix-disklib-distrib /vmware-vix-disklib-distrib RUN mkdir -p /opt ENTRYPOINT ["cp", "-r", "/vmware-vix-disklib-distrib", "/opt"] EOFBuild the image:
$ podman build . -t <registry_route_or_server_path>/vddk:<tag>1 - 1
- Specify your image registry:
-
For an internal OpenShift Container Platform registry, use the internal registry route, for example, .
image-registry.openshift-image-registry.svc:5000/openshift/vddk:<tag> -
For an external registry, specify the server name, path, and tag, for example, .
server.example.com:5000/vddk:<tag>
-
For an internal OpenShift Container Platform registry, use the internal registry route, for example,
Push the image to the registry:
$ podman push <registry_route_or_server_path>/vddk:<tag>- Ensure that the image is accessible to your OpenShift Virtualization environment.
Edit the
CR in the openshift-cnv project:HyperConverged$ oc edit hco -n openshift-cnv kubevirt-hyperconvergedAdd the
parameter to thevddkInitImagestanza:specapiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: vddkInitImage: <registry_route_or_server_path>/vddk:<tag>
8.15.5.3. Importing a virtual machine with the VM Import wizard Copiar enlaceEnlace copiado en el portapapeles!
You can import a single virtual machine with the VM Import wizard.
You can also import a VM template. If you import a VM template, OpenShift Virtualization creates a virtual machine based on the template.
Prerequisites
- You must have admin user privileges.
- The VMware Virtual Disk Development Kit (VDDK) image must be in an image registry that is accessible to your OpenShift Virtualization environment.
-
The VDDK image must be added to the field of the
spec.vddkInitImagecustom resource (CR).HyperConverged - The VM must be powered off.
- Virtual disks must be connected to IDE or SCSI controllers. If virtual disks are connected to a SATA controller, you can change them to IDE controllers and then migrate the VM.
- The OpenShift Virtualization local and shared persistent storage classes must support VM import.
The OpenShift Virtualization storage must be large enough to accommodate the virtual disk.
WarningIf you are using Ceph RBD block-mode volumes, the storage must be large enough to accommodate the virtual disk. If the disk is too large for the available storage, the import process fails and the PV that is used to copy the virtual disk is not released. You will not be able to import another virtual machine or to clean up the storage because there are insufficient resources to support object deletion. To resolve this situation, you must add more object storage devices to the storage back end.
The OpenShift Virtualization egress network policy must allow the following traffic:
Expand Destination Protocol Port VMware ESXi hosts
TCP
443
VMware ESXi hosts
TCP
902
VMware vCenter
TCP
5840
Procedure
- In the web console, click Workloads → Virtual Machines.
- Click Create Virtual Machine and select Import with Wizard.
- Select VMware from the Provider list.
Select Connect to New Instance or a saved vCenter instance.
- If you select Connect to New Instance, enter the vCenter hostname, Username, and Password.
- If you select a saved vCenter instance, the wizard connects to the vCenter instance using the saved credentials.
Click Check and Save and wait for the connection to complete.
NoteThe connection details are stored in a secret. If you add a provider with an incorrect hostname, user name, or password, click Workloads → Secrets and delete the provider secret.
- Select a virtual machine or a template.
- Click Next.
- In the Review screen, review your settings.
Click Edit to update the following settings:
General:
- Description
- Operating System
- Flavor
- Memory
- CPUs
- Workload Profile
Networking:
- Name
- Model
- Network
- Type
- MAC Address
Storage: Click the Options menu
of the VM disk and select Edit to update the following fields:
- Name
- Source: For example, Import Disk.
- Size
- Interface
Storage Class: Select NFS or ocs-storagecluster-ceph-rbd (ceph-rbd).
If you select ocs-storagecluster-ceph-rbd, you must set the Volume Mode of the disk to Block.
Other storage classes might work, but they are not officially supported.
- Advanced → Volume Mode: Select Block.
- Advanced → Access Mode
Advanced → Cloud-init:
- Form: Enter the Hostname and Authenticated SSH Keys.
-
Custom script: Enter the script in the text field.
cloud-init
- Advanced → Virtual Hardware: You can attach a virtual CD-ROM to the imported virtual machine.
Click Import or Review and Import, if you have edited the import settings.
A Successfully created virtual machine message and a list of resources created for the virtual machine are displayed. The virtual machine appears in Workloads → Virtual Machines.
Virtual machine wizard fields
| Name | Parameter | Description |
|---|---|---|
| Name | The name can contain lowercase letters (
| |
| Description | Optional description field. | |
| Operating System | The operating system that is selected for the virtual machine in the template. You cannot edit this field when creating a virtual machine from a template. | |
| Boot Source | Import via URL (creates PVC) | Import content from an image available from an HTTP or HTTPS endpoint. Example: Obtaining a URL link from the web page with the operating system image. |
| Clone existing PVC (creates PVC) | Select an existent persistent volume claim available on the cluster and clone it. | |
| Import via Registry (creates PVC) | Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Example:
| |
| PXE (network boot - adds network interface) | Boot an operating system from a server on the network. Requires a PXE bootable network attachment definition. | |
| Persistent Volume Claim project | Project name that you want to use for cloning the PVC. | |
| Persistent Volume Claim name | PVC name that should apply to this virtual machine template if you are cloning an existing PVC. | |
| Mount this as a CD-ROM boot source | A CD-ROM requires an additional disk for installing the operating system. Select the checkbox to add a disk and customize it later. | |
| Flavor | Tiny, Small, Medium, Large, Custom | Presets the amount of CPU and memory in a virtual machine template with predefined values that are allocated to the virtual machine, depending on the operating system associated with that template. If you choose a default template, you can override the
|
| Workload Type Note If you choose the incorrect Workload Type, there could be performance or resource utilization issues (such as a slow UI). | Desktop | A virtual machine configuration for use on a desktop. Ideal for consumption on a small scale. Recommended for use with the web console. |
| Server | Balances performance and it is compatible with a wide range of server workloads. | |
| High-Performance | A virtual machine configuration that is optimized for high-performance workloads. | |
| Start this virtual machine after creation. | This checkbox is selected by default and the virtual machine starts running after creation. Clear the checkbox if you do not want the virtual machine to start when it is created. |
Cloud-init fields
| Name | Description |
|---|---|
| Hostname | Sets a specific hostname for the virtual machine. |
| Authorized SSH Keys | The user’s public key that is copied to ~/.ssh/authorized_keys on the virtual machine. |
| Custom script | Replaces other options with a field in which you paste a custom cloud-init script. |
Networking fields
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
Storage fields
| Name | Selection | Description |
|---|---|---|
| Source | Blank (creates PVC) | Create an empty disk. |
| Import via URL (creates PVC) | Import content via URL (HTTP or HTTPS endpoint). | |
| Use an existing PVC | Use a PVC that is already available in the cluster. | |
| Clone existing PVC (creates PVC) | Select an existing PVC available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Import content via container registry. | |
| Container (ephemeral) | Upload content from a container located in a registry accessible from the cluster. The container disk should be used only for read-only filesystems such as CD-ROMs or temporary virtual machines. | |
| Name | Name of the disk. The name can contain lowercase letters (
| |
| Size | Size of the disk in GiB. | |
| Type | Type of disk. Example: Disk or CD-ROM | |
| Interface | Type of disk device. Supported interfaces are virtIO, SATA, and SCSI. | |
| Storage Class | The storage class that is used to create the disk. | |
| Advanced → Volume Mode | Defines whether the persistent volume uses a formatted file system or raw block state. Default is Filesystem. | |
| Advanced → Access Mode | Access mode of the persistent volume. Supported access modes are Single User (RWO), Shared Access (RWX), and Read Only (ROX). |
Advanced storage settings
The following advanced storage settings are available for Blank, Import via URL, and Clone existing PVC disks. These parameters are optional. If you do not specify these parameters, the system uses the default values from the
kubevirt-storage-class-defaults
| Name | Parameter | Description |
|---|---|---|
| Volume Mode | Filesystem | Stores the virtual disk on a file system-based volume. |
| Block | Stores the virtual disk directly on the block volume. Only use
| |
| Access Mode | Single User (RWO) | The disk can be mounted as read/write by a single node. |
| Shared Access (RWX) | The disk can be mounted as read/write by many nodes. Note This is required for some features, such as live migration of virtual machines between nodes. | |
| Read Only (ROX) | The disk can be mounted as read-only by many nodes. |
8.15.5.3.1. Updating the NIC name of an imported virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You must update the NIC name of a virtual machine that you imported from VMware to conform to OpenShift Virtualization naming conventions.
Procedure
- Log in to the virtual machine.
-
Navigate to the directory.
/etc/sysconfig/network-scripts Rename the network configuration file:
$ mv vmnic0 ifcfg-eth01 - 1
- The first network configuration file is named
ifcfg-eth0. Additional network configuration files are numbered sequentially, for example,ifcfg-eth1,ifcfg-eth2.
Update the
andNAMEparameters in the network configuration file:DEVICENAME=eth0 DEVICE=eth0Restart the network:
$ systemctl restart network
8.15.5.4. Troubleshooting a virtual machine import Copiar enlaceEnlace copiado en el portapapeles!
8.15.5.4.1. Logs Copiar enlaceEnlace copiado en el portapapeles!
You can check the V2V Conversion pod log for errors.
Procedure
View the V2V Conversion pod name by running the following command:
$ oc get pods -n <namespace> | grep v2v1 - 1
- Specify the namespace of your imported virtual machine.
Example output
kubevirt-v2v-conversion-f66f7d-zqkz7 1/1 Running 0 4h49mView the V2V Conversion pod log by running the following command:
$ oc logs <kubevirt-v2v-conversion-f66f7d-zqkz7> -f -n <namespace>1 - 1
- Specify the VM Conversion pod name and the namespace.
8.15.5.4.2. Error messages Copiar enlaceEnlace copiado en el portapapeles!
The following error messages might appear:
If the VMware VM is not shut down before import, the imported virtual machine displays the error message,
in the OpenShift Container Platform console and the V2V Conversion pod log displays the following error message:Readiness probe failedINFO - have error: ('virt-v2v error: internal error: invalid argument: libvirt domain ‘v2v_migration_vm_1’ is running or paused. It must be shut down in order to perform virt-v2v conversion',)"The following error message is displayed in the OpenShift Container Platform console if a non-admin user tries to import a VM:
Could not load config map vmware-to-kubevirt-os in kube-public namespace Restricted Access: configmaps "vmware-to-kubevirt-os" is forbidden: User cannot get resource "configmaps" in API group "" in the namespace "kube-public"Only an admin user can import a VM.
8.16. Cloning virtual machines Copiar enlaceEnlace copiado en el portapapeles!
8.16.1. Enabling user permissions to clone data volumes across namespaces Copiar enlaceEnlace copiado en el portapapeles!
The isolating nature of namespaces means that users cannot by default clone resources between namespaces.
To enable a user to clone a virtual machine to another namespace, a user with the
cluster-admin
8.16.1.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
Only a user with the
cluster-adminrole can create cluster roles.
8.16.1.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.16.1.3. Creating RBAC resources for cloning data volumes Copiar enlaceEnlace copiado en el portapapeles!
Create a new cluster role that enables permissions for all actions for the
datavolumes
Procedure
Create a
manifest:ClusterRoleapiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: <datavolume-cloner>1 rules: - apiGroups: ["cdi.kubevirt.io"] resources: ["datavolumes/source"] verbs: ["*"]- 1
- Unique name for the cluster role.
Create the cluster role in the cluster:
$ oc create -f <datavolume-cloner.yaml>1 - 1
- The file name of the
ClusterRolemanifest created in the previous step.
Create a
manifest that applies to both the source and destination namespaces and references the cluster role created in the previous step.RoleBindingapiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: <allow-clone-to-user>1 namespace: <Source namespace>2 subjects: - kind: ServiceAccount name: default namespace: <Destination namespace>3 roleRef: kind: ClusterRole name: datavolume-cloner4 apiGroup: rbac.authorization.k8s.ioCreate the role binding in the cluster:
$ oc create -f <datavolume-cloner.yaml>1 - 1
- The file name of the
RoleBindingmanifest created in the previous step.
8.16.2. Cloning a virtual machine disk into a new data volume Copiar enlaceEnlace copiado en el portapapeles!
You can clone the persistent volume claim (PVC) of a virtual machine disk into a new data volume by referencing the source PVC in your data volume configuration file.
Cloning operations between different volume modes are supported, such as cloning from a persistent volume (PV) with
volumeMode: Block
volumeMode: Filesystem
However, you can only clone between different volume modes if they are of the
contentType: kubevirt
When you enable preallocation globally, or for a single data volume, the Containerized Data Importer (CDI) preallocates disk space during cloning. Preallocation enhances write performance. For more information, see Using preallocation for data volumes.
8.16.2.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Users need additional permissions to clone the PVC of a virtual machine disk into another namespace.
8.16.2.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.16.2.3. Cloning the persistent volume claim of a virtual machine disk into a new data volume Copiar enlaceEnlace copiado en el portapapeles!
You can clone a persistent volume claim (PVC) of an existing virtual machine disk into a new data volume. The new data volume can then be used for a new virtual machine.
When a data volume is created independently of a virtual machine, the lifecycle of the data volume is independent of the virtual machine. If the virtual machine is deleted, neither the data volume nor its associated PVC is deleted.
Prerequisites
- Determine the PVC of an existing virtual machine disk to use. You must power down the virtual machine that is associated with the PVC before you can clone it.
-
Install the OpenShift CLI ().
oc
Procedure
- Examine the virtual machine disk you want to clone to identify the name and namespace of the associated PVC.
Create a YAML file for a data volume that specifies the name of the new data volume, the name and namespace of the source PVC, and the size of the new data volume.
For example:
apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <cloner-datavolume>1 spec: source: pvc: namespace: "<source-namespace>"2 name: "<my-favorite-vm-disk>"3 pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi>4 Start cloning the PVC by creating the data volume:
$ oc create -f <cloner-datavolume>.yamlNoteData volumes prevent a virtual machine from starting before the PVC is prepared, so you can create a virtual machine that references the new data volume while the PVC clones.
8.16.2.4. Template: Data volume clone configuration file Copiar enlaceEnlace copiado en el portapapeles!
example-clone-dv.yaml
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: "example-clone-dv"
spec:
source:
pvc:
name: source-pvc
namespace: example-ns
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: "1G"
8.16.2.5. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.16.3. Cloning a virtual machine by using a data volume template Copiar enlaceEnlace copiado en el portapapeles!
You can create a new virtual machine by cloning the persistent volume claim (PVC) of an existing VM. By including a
dataVolumeTemplate
Cloning operations between different volume modes are supported, such as cloning from a persistent volume (PV) with
volumeMode: Block
volumeMode: Filesystem
However, you can only clone between different volume modes if they are of the
contentType: kubevirt
When you enable preallocation globally, or for a single data volume, the Containerized Data Importer (CDI) preallocates disk space during cloning. Preallocation enhances write performance. For more information, see Using preallocation for data volumes.
8.16.3.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Users need additional permissions to clone the PVC of a virtual machine disk into another namespace.
8.16.3.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.16.3.3. Creating a new virtual machine from a cloned persistent volume claim by using a data volume template Copiar enlaceEnlace copiado en el portapapeles!
You can create a virtual machine that clones the persistent volume claim (PVC) of an existing virtual machine into a data volume. Reference a
dataVolumeTemplate
source
When a data volume is created as part of the data volume template of a virtual machine, the lifecycle of the data volume is then dependent on the virtual machine. If the virtual machine is deleted, the data volume and associated PVC are also deleted.
Prerequisites
- Determine the PVC of an existing virtual machine disk to use. You must power down the virtual machine that is associated with the PVC before you can clone it.
-
Install the OpenShift CLI ().
oc
Procedure
- Examine the virtual machine you want to clone to identify the name and namespace of the associated PVC.
Create a YAML file for a
object. The following virtual machine example clonesVirtualMachine, which is located in themy-favorite-vm-disknamespace. Thesource-namespacedata volume called2Giis created fromfavorite-clone.my-favorite-vm-diskFor example:
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: labels: kubevirt.io/vm: vm-dv-clone name: vm-dv-clone1 spec: running: false template: metadata: labels: kubevirt.io/vm: vm-dv-clone spec: domain: devices: disks: - disk: bus: virtio name: root-disk resources: requests: memory: 64M volumes: - dataVolume: name: favorite-clone name: root-disk dataVolumeTemplates: - metadata: name: favorite-clone spec: pvc: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi source: pvc: namespace: "source-namespace" name: "my-favorite-vm-disk"- 1
- The virtual machine to create.
Create the virtual machine with the PVC-cloned data volume:
$ oc create -f <vm-clone-datavolumetemplate>.yaml
8.16.3.4. Template: Data volume virtual machine configuration file Copiar enlaceEnlace copiado en el portapapeles!
example-dv-vm.yaml
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
labels:
kubevirt.io/vm: example-vm
name: example-vm
spec:
dataVolumeTemplates:
- metadata:
name: example-dv
spec:
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1G
source:
http:
url: ""
running: false
template:
metadata:
labels:
kubevirt.io/vm: example-vm
spec:
domain:
cpu:
cores: 1
devices:
disks:
- disk:
bus: virtio
name: example-dv-disk
machine:
type: q35
resources:
requests:
memory: 1G
terminationGracePeriodSeconds: 0
volumes:
- dataVolume:
name: example-dv
name: example-dv-disk
- 1
- The
HTTPsource of the image you want to import, if applicable.
8.16.3.5. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.16.4. Cloning a virtual machine disk into a new block storage data volume Copiar enlaceEnlace copiado en el portapapeles!
You can clone the persistent volume claim (PVC) of a virtual machine disk into a new block data volume by referencing the source PVC in your data volume configuration file.
Cloning operations between different volume modes are supported, such as cloning from a persistent volume (PV) with
volumeMode: Block
volumeMode: Filesystem
However, you can only clone between different volume modes if they are of the
contentType: kubevirt
When you enable preallocation globally, or for a single data volume, the Containerized Data Importer (CDI) preallocates disk space during cloning. Preallocation enhances write performance. For more information, see Using preallocation for data volumes.
8.16.4.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- Users need additional permissions to clone the PVC of a virtual machine disk into another namespace.
8.16.4.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.16.4.3. About block persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
A block persistent volume (PV) is a PV that is backed by a raw block device. These volumes do not have a file system and can provide performance benefits for virtual machines by reducing overhead.
Raw block volumes are provisioned by specifying
volumeMode: Block
8.16.4.4. Creating a local block persistent volume Copiar enlaceEnlace copiado en el portapapeles!
Create a local block persistent volume (PV) on a node by populating a file and mounting it as a loop device. You can then reference this loop device in a PV manifest as a
Block
Procedure
-
Log in as to the node on which to create the local PV. This procedure uses
rootfor its examples.node01 Create a file and populate it with null characters so that it can be used as a block device. The following example creates a file
with a size of 2Gb (20 100Mb blocks):loop10$ dd if=/dev/zero of=<loop10> bs=100M count=20Mount the
file as a loop device.loop10$ losetup </dev/loop10>d3 <loop10>1 2 Create a
manifest that references the mounted loop device.PersistentVolumekind: PersistentVolume apiVersion: v1 metadata: name: <local-block-pv10> annotations: spec: local: path: </dev/loop10>1 capacity: storage: <2Gi> volumeMode: Block2 storageClassName: local3 accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - <node01>4 Create the block PV.
# oc create -f <local-block-pv10.yaml>1 - 1
- The file name of the persistent volume created in the previous step.
8.16.4.5. Cloning the persistent volume claim of a virtual machine disk into a new data volume Copiar enlaceEnlace copiado en el portapapeles!
You can clone a persistent volume claim (PVC) of an existing virtual machine disk into a new data volume. The new data volume can then be used for a new virtual machine.
When a data volume is created independently of a virtual machine, the lifecycle of the data volume is independent of the virtual machine. If the virtual machine is deleted, neither the data volume nor its associated PVC is deleted.
Prerequisites
- Determine the PVC of an existing virtual machine disk to use. You must power down the virtual machine that is associated with the PVC before you can clone it.
-
Install the OpenShift CLI ().
oc - At least one available block persistent volume (PV) that is the same size as or larger than the source PVC.
Procedure
- Examine the virtual machine disk you want to clone to identify the name and namespace of the associated PVC.
Create a YAML file for a data volume that specifies the name of the new data volume, the name and namespace of the source PVC,
so that an available block PV is used, and the size of the new data volume.volumeMode: BlockFor example:
apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <cloner-datavolume>1 spec: source: pvc: namespace: "<source-namespace>"2 name: "<my-favorite-vm-disk>"3 pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi>4 volumeMode: Block5 - 1
- The name of the new data volume.
- 2
- The namespace where the source PVC exists.
- 3
- The name of the source PVC.
- 4
- The size of the new data volume. You must allocate enough space, or the cloning operation fails. The size must be the same as or larger than the source PVC.
- 5
- Specifies that the destination is a block PV
Start cloning the PVC by creating the data volume:
$ oc create -f <cloner-datavolume>.yamlNoteData volumes prevent a virtual machine from starting before the PVC is prepared, so you can create a virtual machine that references the new data volume while the PVC clones.
8.16.4.6. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.17. Virtual machine networking Copiar enlaceEnlace copiado en el portapapeles!
8.17.1. Configuring the virtual machine for the default pod network Copiar enlaceEnlace copiado en el portapapeles!
You can connect a virtual machine to the default internal pod network by configuring its network interface to use the
masquerade
8.17.1.1. Configuring masquerade mode from the command line Copiar enlaceEnlace copiado en el portapapeles!
You can use masquerade mode to hide a virtual machine’s outgoing traffic behind the pod IP address. Masquerade mode uses Network Address Translation (NAT) to connect virtual machines to the pod network backend through a Linux bridge.
Enable masquerade mode and allow traffic to enter the virtual machine by editing your virtual machine configuration file.
Prerequisites
- The virtual machine must be configured to use DHCP to acquire IPv4 addresses. The examples below are configured to use DHCP.
Procedure
Edit the
spec of your virtual machine configuration file:interfaceskind: VirtualMachine spec: domain: devices: interfaces: - name: default masquerade: {}1 ports:2 - port: 80 networks: - name: default pod: {}- 1
- Connect using masquerade mode.
- 2
- Optional: List the ports that you want to expose from the virtual machine, each specified by the
portfield. Theportvalue must be a number between 0 and 65536. When theportsarray is not used, all ports in the valid range are open to incoming traffic. In this example, incoming traffic is allowed on port80.
NotePorts 49152 and 49153 are reserved for use by the libvirt platform and all other incoming traffic to these ports is dropped.
Create the virtual machine:
$ oc create -f <vm-name>.yaml
8.17.1.2. Configuring masquerade mode with dual-stack (IPv4 and IPv6) Copiar enlaceEnlace copiado en el portapapeles!
You can configure a new virtual machine to use both IPv6 and IPv4 on the default pod network by using cloud-init.
The IPv6 network address must be statically set to
fd10:0:2::2/120
fd10:0:2::1
When the virtual machine is running, incoming and outgoing traffic for the virtual machine is routed to both the IPv4 address and the unique IPv6 address of the virt-launcher pod. The virt-launcher pod then routes the IPv4 traffic to the DHCP address of the virtual machine, and the IPv6 traffic to the statically set IPv6 address of the virtual machine.
Prerequisites
- The OpenShift Container Platform cluster must use the OVN-Kubernetes Container Network Interface (CNI) network provider configured for dual-stack.
Procedure
In a new virtual machine configuration, include an interface with
and configure the IPv6 address and default gateway by using cloud-init.masqueradeapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: example-vm-ipv6 ... interfaces: - name: default masquerade: {}1 ports: - port: 802 networks: - name: default pod: {} volumes: - cloudInitNoCloud: networkData: | version: 2 ethernets: eth0: dhcp4: true addresses: [ fd10:0:2::2/120 ]3 gateway6: fd10:0:2::14 Create the virtual machine in the namespace:
$ oc create -f example-vm-ipv6.yaml
Verification
- To verify that IPv6 has been configured, start the virtual machine and view the interface status of the virtual machine instance to ensure it has an IPv6 address:
$ oc get vmi <vmi-name> -o jsonpath="{.status.interfaces[*].ipAddresses}"
8.17.2. Creating a service to expose a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can expose a virtual machine within the cluster or outside the cluster by using a
Service
8.17.2.1. About services Copiar enlaceEnlace copiado en el portapapeles!
A Kubernetes service is an abstract way to expose an application running on a set of pods as a network service. Services allow your applications to receive traffic. Services can be exposed in different ways by specifying a
spec.type
Service
- ClusterIP
-
Exposes the service on an internal IP address within the cluster.
ClusterIPis the default servicetype. - NodePort
-
Exposes the service on the same port of each selected node in the cluster.
NodePortmakes a service accessible from outside the cluster. - LoadBalancer
- Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP address to the service.
8.17.2.1.1. Dual-stack support Copiar enlaceEnlace copiado en el portapapeles!
If IPv4 and IPv6 dual-stack networking is enabled for your cluster, you can create a service that uses IPv4, IPv6, or both, by defining the
spec.ipFamilyPolicy
spec.ipFamilies
Service
The
spec.ipFamilyPolicy
- SingleStack
- The control plane assigns a cluster IP address for the service based on the first configured service cluster IP range.
- PreferDualStack
- The control plane assigns both IPv4 and IPv6 cluster IP addresses for the service on clusters that have dual-stack configured.
- RequireDualStack
-
This option fails for clusters that do not have dual-stack networking enabled. For clusters that have dual-stack configured, the behavior is the same as when the value is set to
PreferDualStack. The control plane allocates cluster IP addresses from both IPv4 and IPv6 address ranges.
You can define which IP family to use for single-stack or define the order of IP families for dual-stack by setting the
spec.ipFamilies
-
[IPv4] -
[IPv6] -
[IPv4, IPv6] -
[IPv6, IPv4]
8.17.2.2. Exposing a virtual machine as a service Copiar enlaceEnlace copiado en el portapapeles!
Create a
ClusterIP
NodePort
LoadBalancer
Procedure
Edit the
manifest to add the label for service creation:VirtualMachineapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: vm-ephemeral namespace: example-namespace spec: running: false template: metadata: labels: special: key1 # ...- 1
- Add the label
special: keyin thespec.template.metadata.labelssection.
NoteLabels on a virtual machine are passed through to the pod. The
label must match the label in thespecial: keyattribute of thespec.selectormanifest.Service-
Save the manifest file to apply your changes.
VirtualMachine Create a
manifest to expose the VM:ServiceapiVersion: v1 kind: Service metadata: name: vmservice1 namespace: example-namespace2 spec: externalTrafficPolicy: Cluster3 ports: - nodePort: 300004 port: 27017 protocol: TCP targetPort: 225 selector: special: key6 type: NodePort7 - 1
- The name of the
Serviceobject. - 2
- The namespace where the
Serviceobject resides. This must match themetadata.namespacefield of theVirtualMachinemanifest. - 3
- Optional: Specifies how the nodes distribute service traffic that is received on external IP addresses. This only applies to
NodePortandLoadBalancerservice types. The default value isClusterwhich routes traffic evenly to all cluster endpoints. - 4
- Optional: When set, the
nodePortvalue must be unique across all services. If not specified, a value in the range above30000is dynamically allocated. - 5
- Optional: The VM port to be exposed by the service. It must reference an open port if a port list is defined in the VM manifest. If
targetPortis not specified, it takes the same value asport. - 6
- The reference to the label that you added in the
spec.template.metadata.labelsstanza of theVirtualMachinemanifest. - 7
- The type of service. Possible values are
ClusterIP,NodePortandLoadBalancer.
-
Save the manifest file.
Service Create the service by running the following command:
$ oc create -f <service_name>.yaml- Start the VM. If the VM is already running, restart it.
Verification
Query the
object to verify that it is available:Service$ oc get service -n example-namespaceExample output for
ClusterIPserviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE vmservice ClusterIP 172.30.3.149 <none> 27017/TCP 2mExample output for
NodePortserviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE vmservice NodePort 172.30.232.73 <none> 27017:30000/TCP 5mExample output for
LoadBalancerserviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE vmservice LoadBalancer 172.30.27.5 172.29.10.235,172.29.10.235 27017:31829/TCP 5sChoose the appropriate method to connect to the virtual machine:
For a
service, connect to the VM from within the cluster by using the service IP address and the service port. For example:ClusterIP$ ssh fedora@172.30.3.149 -p 27017For a
service, connect to the VM by specifying the node IP address and the node port outside the cluster network. For example:NodePort$ ssh fedora@$NODE_IP -p 30000-
For a service, use the
LoadBalancerclient to connect to your virtual machine by using the public IP address and port. External ports are dynamically allocated.vinagre
8.17.3. Attaching a virtual machine to a Linux bridge network Copiar enlaceEnlace copiado en el portapapeles!
By default, OpenShift Virtualization is installed with a single, internal pod network.
You must create a Linux bridge network attachment definition (NAD) in order to connect to additional networks.
To attach a virtual machine to an additional network:
- Create a Linux bridge node network configuration policy.
- Create a Linux bridge network attachment definition.
- Configure the virtual machine, enabling the virtual machine to recognize the network attachment definition.
For more information about scheduling, interface types, and other node networking activities, see the node networking section.
8.17.3.1. Connecting to the network through the network attachment definition Copiar enlaceEnlace copiado en el portapapeles!
8.17.3.1.1. Creating a Linux bridge node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
Use a
NodeNetworkConfigurationPolicy
Procedure
Create the
manifest. This example includes sample values that you must replace with your own information.NodeNetworkConfigurationPolicyapiVersion: nmstate.io/v1 kind: NodeNetworkConfigurationPolicy metadata: name: br1-eth1-policy1 spec: desiredState: interfaces: - name: br12 description: Linux bridge with eth1 as a port3 type: linux-bridge4 state: up5 ipv4: enabled: false6 bridge: options: stp: enabled: false7 port: - name: eth18 - 1
- Name of the policy.
- 2
- Name of the interface.
- 3
- Optional: Human-readable description of the interface.
- 4
- The type of interface. This example creates a bridge.
- 5
- The requested state for the interface after creation.
- 6
- Disables IPv4 in this example.
- 7
- Disables STP in this example.
- 8
- The node NIC to which the bridge is attached.
8.17.3.2. Creating a Linux bridge network attachment definition Copiar enlaceEnlace copiado en el portapapeles!
8.17.3.2.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- A Linux bridge must be configured and attached on every node. See the node networking section for more information.
Configuring IP address management (IPAM) in a network attachment definition for virtual machines is not supported.
8.17.3.2.2. Creating a Linux bridge network attachment definition in the web console Copiar enlaceEnlace copiado en el portapapeles!
The network attachment definition is a custom resource that exposes layer-2 devices to a specific namespace in your OpenShift Virtualization cluster.
Network administrators can create network attachment definitions to provide existing layer-2 networking to pods and virtual machines.
Procedure
- In the web console, click Networking → Network Attachment Definitions.
Click Create Network Attachment Definition.
NoteThe network attachment definition must be in the same namespace as the pod or virtual machine.
- Enter a unique Name and optional Description.
- Click the Network Type list and select CNV Linux bridge.
- Enter the name of the bridge in the Bridge Name field.
- Optional: If the resource has VLAN IDs configured, enter the ID numbers in the VLAN Tag Number field.
- Optional: Select MAC Spoof Check to enable MAC spoof filtering. This feature provides security against a MAC spoofing attack by allowing only a single MAC address to exit the pod.
Click Create.
NoteA Linux bridge network attachment definition is the most efficient method for connecting a virtual machine to a VLAN.
8.17.3.2.3. Creating a Linux bridge network attachment definition in the CLI Copiar enlaceEnlace copiado en el portapapeles!
As a network administrator, you can configure a network attachment definition of type
cnv-bridge
The network attachment definition must be in the same namespace as the pod or virtual machine.
Procedure
- Create a network attachment definition in the same namespace as the virtual machine.
Add the virtual machine to the network attachment definition, as in the following example:
apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: <bridge-network>1 annotations: k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/<bridge-interface>2 spec: config: '{ "cniVersion": "0.3.1", "name": "<bridge-network>",3 "type": "cnv-bridge",4 "bridge": "<bridge-interface>",5 "macspoofchk": true,6 "vlan": 17 }'- 1
- The name for the
NetworkAttachmentDefinitionobject. - 2
- Optional: Annotation key-value pair for node selection, where
bridge-interfaceis the name of a bridge configured on some nodes. If you add this annotation to your network attachment definition, your virtual machine instances will only run on the nodes that have thebridge-interfacebridge connected. - 3
- The name for the configuration. It is recommended to match the configuration name to the
namevalue of the network attachment definition. - 4
- The actual name of the Container Network Interface (CNI) plugin that provides the network for this network attachment definition. Do not change this field unless you want to use a different CNI.
- 5
- The name of the Linux bridge configured on the node.
- 6
- Optional: Flag to enable MAC spoof check. When set to
true, you cannot change the MAC address of the pod or guest interface. This attribute provides security against a MAC spoofing attack by allowing only a single MAC address to exit the pod. - 7
- Optional: The VLAN tag. No additional VLAN configuration is required on the node network configuration policy.
NoteA Linux bridge network attachment definition is the most efficient method for connecting a virtual machine to a VLAN.
Create the network attachment definition:
$ oc create -f <network-attachment-definition.yaml>1 - 1
- Where
<network-attachment-definition.yaml>is the file name of the network attachment definition manifest.
Verification
Verify that the network attachment definition was created by running the following command:
$ oc get network-attachment-definition <bridge-network>
8.17.3.3. Configuring the virtual machine for a Linux bridge network Copiar enlaceEnlace copiado en el portapapeles!
8.17.3.3.1. Creating a NIC for a virtual machine in the web console Copiar enlaceEnlace copiado en el portapapeles!
Create and attach additional NICs to a virtual machine from the web console.
Procedure
- In the correct project in the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click Network Interfaces to display the NICs already attached to the virtual machine.
- Click Add Network Interface to create a new slot in the list.
- Use the Network drop-down list to select the network attachment definition for the additional network.
- Fill in the Name, Model, Type, and MAC Address for the new NIC.
- Click Add to save and attach the NIC to the virtual machine.
8.17.3.3.2. Networking fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
8.17.3.3.3. Attaching a virtual machine to an additional network in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Attach a virtual machine to an additional network by adding a bridge interface and specifying a network attachment definition in the virtual machine configuration.
This procedure uses a YAML file to demonstrate editing the configuration and applying the updated file to the cluster. You can alternatively use the
oc edit <object> <name>
Prerequisites
- Shut down the virtual machine before editing the configuration. If you edit a running virtual machine, you must restart the virtual machine for the changes to take effect.
Procedure
- Create or edit a configuration of a virtual machine that you want to connect to the bridge network.
Add the bridge interface to the
list and the network attachment definition to thespec.template.spec.domain.devices.interfaceslist. This example adds a bridge interface calledspec.template.spec.networksthat connects to thebridge-netnetwork attachment definition:a-bridge-networkapiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: <example-vm> spec: template: spec: domain: devices: interfaces: - masquerade: {} name: <default> - bridge: {} name: <bridge-net>1 ... networks: - name: <default> pod: {} - name: <bridge-net>2 multus: networkName: <network-namespace>/<a-bridge-network>3 ...- 1
- The name of the bridge interface.
- 2
- The name of the network. This value must match the
namevalue of the correspondingspec.template.spec.domain.devices.interfacesentry. - 3
- The name of the network attachment definition, prefixed by the namespace where it exists. The namespace must be either the
defaultnamespace or the same namespace where the VM is to be created. In this case,multusis used. Multus is a cloud network interface (CNI) plugin that allows multiple CNIs to exist so that a pod or virtual machine can use the interfaces it needs.
Apply the configuration:
$ oc apply -f <example-vm.yaml>- Optional: If you edited a running virtual machine, you must restart it for the changes to take effect.
8.17.4. Configuring IP addresses for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can configure either dynamically or statically provisioned IP addresses for virtual machines.
Prerequisites
- The virtual machine must connect to an external network.
- You must have a DHCP server available on the additional network to configure a dynamic IP for the virtual machine.
8.17.4.1. Configuring an IP address for a new virtual machine using cloud-init Copiar enlaceEnlace copiado en el portapapeles!
You can use cloud-init to configure an IP address when you create a virtual machine. The IP address can be dynamically or statically provisioned.
Procedure
Create a virtual machine configuration and include the cloud-init network details in the
field of the virtual machine configuration:spec.volumes.cloudInitNoCloud.networkDataTo configure a dynamic IP, specify the interface name and the
boolean:dhcp4kind: VirtualMachine spec: ... volumes: - cloudInitNoCloud: networkData: | version: 2 ethernets: eth1:1 dhcp4: true2 To configure a static IP, specify the interface name and the IP address:
kind: VirtualMachine spec: ... volumes: - cloudInitNoCloud: networkData: | version: 2 ethernets: eth1:1 addresses: - 10.10.10.14/242
8.17.5. Configuring an SR-IOV network device for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can configure a Single Root I/O Virtualization (SR-IOV) device for virtual machines in your cluster. This process is similar but not identical to configuring an SR-IOV device for OpenShift Container Platform.
Live migration is supported for virtual machines that are attached to an SR-IOV network interface only if the
sriovLiveMigration
spec.featureGates.sriovLiveMigration
true
virt-launcher
SYS_RESOURCE
8.17.5.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have installed the SR-IOV Operator.
- You must have configured the SR-IOV Operator.
8.17.5.2. Automated discovery of SR-IOV network devices Copiar enlaceEnlace copiado en el portapapeles!
The SR-IOV Network Operator searches your cluster for SR-IOV capable network devices on worker nodes. The Operator creates and updates a SriovNetworkNodeState custom resource (CR) for each worker node that provides a compatible SR-IOV network device.
The CR is assigned the same name as the worker node. The
status.interfaces
Do not modify a
SriovNetworkNodeState
8.17.5.2.1. Example SriovNetworkNodeState object Copiar enlaceEnlace copiado en el portapapeles!
The following YAML is an example of a
SriovNetworkNodeState
An SriovNetworkNodeState object
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodeState
metadata:
name: node-25
namespace: openshift-sriov-network-operator
ownerReferences:
- apiVersion: sriovnetwork.openshift.io/v1
blockOwnerDeletion: true
controller: true
kind: SriovNetworkNodePolicy
name: default
spec:
dpConfigVersion: "39824"
status:
interfaces:
- deviceID: "1017"
driver: mlx5_core
mtu: 1500
name: ens785f0
pciAddress: "0000:18:00.0"
totalvfs: 8
vendor: 15b3
- deviceID: "1017"
driver: mlx5_core
mtu: 1500
name: ens785f1
pciAddress: "0000:18:00.1"
totalvfs: 8
vendor: 15b3
- deviceID: 158b
driver: i40e
mtu: 1500
name: ens817f0
pciAddress: 0000:81:00.0
totalvfs: 64
vendor: "8086"
- deviceID: 158b
driver: i40e
mtu: 1500
name: ens817f1
pciAddress: 0000:81:00.1
totalvfs: 64
vendor: "8086"
- deviceID: 158b
driver: i40e
mtu: 1500
name: ens803f0
pciAddress: 0000:86:00.0
totalvfs: 64
vendor: "8086"
syncStatus: Succeeded
8.17.5.3. Configuring SR-IOV network devices Copiar enlaceEnlace copiado en el portapapeles!
The SR-IOV Network Operator adds the
SriovNetworkNodePolicy.sriovnetwork.openshift.io
When applying the configuration specified in a
SriovNetworkNodePolicy
It might take several minutes for a configuration change to apply.
Prerequisites
-
You installed the OpenShift CLI ().
oc -
You have access to the cluster as a user with the role.
cluster-admin - You have installed the SR-IOV Network Operator.
- You have enough available nodes in your cluster to handle the evicted workload from drained nodes.
- You have not selected any control plane nodes for SR-IOV network device configuration.
Procedure
Create an
object, and then save the YAML in theSriovNetworkNodePolicyfile. Replace<name>-sriov-node-network.yamlwith the name for this configuration.<name>apiVersion: sriovnetwork.openshift.io/v1 kind: SriovNetworkNodePolicy metadata: name: <name>1 namespace: openshift-sriov-network-operator2 spec: resourceName: <sriov_resource_name>3 nodeSelector: feature.node.kubernetes.io/network-sriov.capable: "true"4 priority: <priority>5 mtu: <mtu>6 numVfs: <num>7 nicSelector:8 vendor: "<vendor_code>"9 deviceID: "<device_id>"10 pfNames: ["<pf_name>", ...]11 rootDevices: ["<pci_bus_id>", "..."]12 deviceType: vfio-pci13 isRdma: false14 - 1
- Specify a name for the CR object.
- 2
- Specify the namespace where the SR-IOV Operator is installed.
- 3
- Specify the resource name of the SR-IOV device plugin. You can create multiple
SriovNetworkNodePolicyobjects for a resource name. - 4
- Specify the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.
- 5
- Optional: Specify an integer value between
0and99. A smaller number gets higher priority, so a priority of10is higher than a priority of99. The default value is99. - 6
- Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.
- 7
- Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than
128. - 8
- The
nicSelectormapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters. It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally. If you specifyrootDevices, you must also specify a value forvendor,deviceID, orpfNames. If you specify bothpfNamesandrootDevicesat the same time, ensure that they point to an identical device. - 9
- Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either
8086or15b3. - 10
- Optional: Specify the device hex code of SR-IOV network device. The only allowed values are
158b,1015,1017. - 11
- Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
- 12
- The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format:
0000:02:00.1. - 13
- The
vfio-pcidriver type is required for virtual functions in OpenShift Virtualization. - 14
- Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set
isRdmatofalse. The default value isfalse.
NoteIf
flag is set toisRDMA, you can continue to use the RDMA enabled VF as a normal network device. A device can be used in either mode.true-
Optional: Label the SR-IOV capable cluster nodes with if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".
SriovNetworkNodePolicy.Spec.NodeSelector Create the
object:SriovNetworkNodePolicy$ oc create -f <name>-sriov-node-network.yamlwhere
specifies the name for this configuration.<name>After applying the configuration update, all the pods in
namespace transition to thesriov-network-operatorstatus.RunningTo verify that the SR-IOV network device is configured, enter the following command. Replace
with the name of a node with the SR-IOV network device that you just configured.<node_name>$ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'
8.17.5.4. Next steps Copiar enlaceEnlace copiado en el portapapeles!
8.17.6. Defining an SR-IOV network Copiar enlaceEnlace copiado en el portapapeles!
You can create a network attachment for a Single Root I/O Virtualization (SR-IOV) device for virtual machines.
After the network is defined, you can attach virtual machines to the SR-IOV network.
8.17.6.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have configured an SR-IOV device for virtual machines.
8.17.6.2. Configuring SR-IOV additional network Copiar enlaceEnlace copiado en el portapapeles!
You can configure an additional network that uses SR-IOV hardware by creating a
SriovNetwork
SriovNetwork
NetworkAttachmentDefinition
Users can then attach virtual machines to the SR-IOV network by specifying the network in the virtual machine configurations.
Do not modify or delete a
SriovNetwork
running
Prerequisites
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
Procedure
-
Create the following object, and then save the YAML in the
SriovNetworkfile. Replace<name>-sriov-network.yamlwith a name for this additional network.<name>
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
name: <name>
namespace: openshift-sriov-network-operator
spec:
resourceName: <sriov_resource_name>
networkNamespace: <target_namespace>
vlan: <vlan>
spoofChk: "<spoof_check>"
linkState: <link_state>
maxTxRate: <max_tx_rate>
minTxRate: <min_rx_rate>
vlanQoS: <vlan_qos>
trust: "<trust_vf>"
capabilities: <capabilities>
- 1
- Replace
<name>with a name for the object. The SR-IOV Network Operator creates aNetworkAttachmentDefinitionobject with same name. - 2
- Specify the namespace where the SR-IOV Network Operator is installed.
- 3
- Replace
<sriov_resource_name>with the value for the.spec.resourceNameparameter from theSriovNetworkNodePolicyobject that defines the SR-IOV hardware for this additional network. - 4
- Replace
<target_namespace>with the target namespace for the SriovNetwork. Only pods or virtual machines in the target namespace can attach to the SriovNetwork. - 5
- Optional: Replace
<vlan>with a Virtual LAN (VLAN) ID for the additional network. The integer value must be from0to4095. The default value is0. - 6
- Optional: Replace
<spoof_check>with the spoof check mode of the VF. The allowed values are the strings"on"and"off".ImportantYou must enclose the value you specify in quotes or the CR is rejected by the SR-IOV Network Operator.
- 7
- Optional: Replace
<link_state>with the link state of virtual function (VF). Allowed value areenable,disableandauto. - 8
- Optional: Replace
<max_tx_rate>with a maximum transmission rate, in Mbps, for the VF. - 9
- Optional: Replace
<min_tx_rate>with a minimum transmission rate, in Mbps, for the VF. This value should always be less than or equal to Maximum transmission rate.Note - 10
- Optional: Replace
<vlan_qos>with an IEEE 802.1p priority level for the VF. The default value is0. - 11
- Optional: Replace
<trust_vf>with the trust mode of the VF. The allowed values are the strings"on"and"off".ImportantYou must enclose the value you specify in quotes or the CR is rejected by the SR-IOV Network Operator.
- 12
- Optional: Replace
<capabilities>with the capabilities to configure for this network.
To create the object, enter the following command. Replace
with a name for this additional network.<name>$ oc create -f <name>-sriov-network.yamlOptional: To confirm that the
object associated with theNetworkAttachmentDefinitionobject that you created in the previous step exists, enter the following command. ReplaceSriovNetworkwith the namespace you specified in the<namespace>object.SriovNetwork$ oc get net-attach-def -n <namespace>
8.17.6.3. Next steps Copiar enlaceEnlace copiado en el portapapeles!
8.17.7. Attaching a virtual machine to an SR-IOV network Copiar enlaceEnlace copiado en el portapapeles!
You can attach a virtual machine to use a Single Root I/O Virtualization (SR-IOV) network as a secondary network.
8.17.7.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have configured an SR-IOV device for virtual machines.
- You must have defined an SR-IOV network.
8.17.7.2. Attaching a virtual machine to an SR-IOV network Copiar enlaceEnlace copiado en el portapapeles!
You can attach the virtual machine to the SR-IOV network by including the network details in the virtual machine configuration.
Procedure
Include the SR-IOV network details in the
andspec.domain.devices.interfacesof the virtual machine configuration:spec.networkskind: VirtualMachine ... spec: domain: devices: interfaces: - name: <default>1 masquerade: {}2 - name: <nic1>3 sriov: {} networks: - name: <default>4 pod: {} - name: <nic1>5 multus: networkName: <sriov-network>6 ...- 1
- A unique name for the interface that is connected to the pod network.
- 2
- The
masqueradebinding to the default pod network. - 3
- A unique name for the SR-IOV interface.
- 4
- The name of the pod network interface. This must be the same as the
interfaces.namethat you defined earlier. - 5
- The name of the SR-IOV interface. This must be the same as the
interfaces.namethat you defined earlier. - 6
- The name of the SR-IOV network attachment definition.
Apply the virtual machine configuration:
$ oc apply -f <vm-sriov.yaml>1 - 1
- The name of the virtual machine YAML file.
8.17.8. Viewing the IP address of NICs on a virtual machine Copiar enlaceEnlace copiado en el portapapeles!
You can view the IP address for a network interface controller (NIC) by using the web console or the
oc
8.17.8.1. Viewing the IP address of a virtual machine interface in the CLI Copiar enlaceEnlace copiado en el portapapeles!
The network interface configuration is included in the
oc describe vmi <vmi_name>
You can also view the IP address information by running
ip addr
oc get vmi <vmi_name> -o yaml
Procedure
Use the
command to display the virtual machine interface configuration:oc describe$ oc describe vmi <vmi_name>Example output
... Interfaces: Interface Name: eth0 Ip Address: 10.244.0.37/24 Ip Addresses: 10.244.0.37/24 fe80::858:aff:fef4:25/64 Mac: 0a:58:0a:f4:00:25 Name: default Interface Name: v2 Ip Address: 1.1.1.7/24 Ip Addresses: 1.1.1.7/24 fe80::f4d9:70ff:fe13:9089/64 Mac: f6:d9:70:13:90:89 Interface Name: v1 Ip Address: 1.1.1.1/24 Ip Addresses: 1.1.1.1/24 1.1.1.2/24 1.1.1.4/24 2001:de7:0:f101::1/64 2001:db8:0:f101::1/64 fe80::1420:84ff:fe10:17aa/64 Mac: 16:20:84:10:17:aa
8.17.8.2. Viewing the IP address of a virtual machine interface in the web console Copiar enlaceEnlace copiado en el portapapeles!
The IP information displays in the Virtual Machine Overview screen for the virtual machine.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine name to open the Virtual Machine Overview screen.
The information for each attached NIC is displayed under IP Address.
8.17.9. Using a MAC address pool for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
The KubeMacPool component provides a MAC address pool service for virtual machine NICs in a namespace.
8.17.9.1. About KubeMacPool Copiar enlaceEnlace copiado en el portapapeles!
KubeMacPool provides a MAC address pool per namespace and allocates MAC addresses for virtual machine NICs from the pool. This ensures that the NIC is assigned a unique MAC address that does not conflict with the MAC address of another virtual machine.
Virtual machine instances created from that virtual machine retain the assigned MAC address across reboots.
KubeMacPool does not handle virtual machine instances created independently from a virtual machine.
KubeMacPool is enabled by default when you install OpenShift Virtualization. You can disable a MAC address pool for a namespace by adding the
mutatevirtualmachines.kubemacpool.io=ignore
8.17.9.2. Disabling a MAC address pool for a namespace in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Disable a MAC address pool for virtual machines in a namespace by adding the
mutatevirtualmachines.kubemacpool.io=ignore
Procedure
Add the
label to the namespace. The following example disables KubeMacPool for two namespaces,mutatevirtualmachines.kubemacpool.io=ignoreand<namespace1>:<namespace2>$ oc label namespace <namespace1> <namespace2> mutatevirtualmachines.kubemacpool.io=ignore
8.17.9.3. Re-enabling a MAC address pool for a namespace in the CLI Copiar enlaceEnlace copiado en el portapapeles!
If you disabled KubeMacPool for a namespace and want to re-enable it, remove the
mutatevirtualmachines.kubemacpool.io=ignore
Earlier versions of OpenShift Virtualization used the label
mutatevirtualmachines.kubemacpool.io=allocate
Procedure
Remove the KubeMacPool label from the namespace. The following example re-enables KubeMacPool for two namespaces,
and<namespace1>:<namespace2>$ oc label namespace <namespace1> <namespace2> mutatevirtualmachines.kubemacpool.io-
8.18. Virtual machine disks Copiar enlaceEnlace copiado en el portapapeles!
8.18.1. Storage features Copiar enlaceEnlace copiado en el portapapeles!
Use the following table to determine feature availability for local and shared persistent storage in OpenShift Virtualization.
8.18.1.1. OpenShift Virtualization storage feature matrix Copiar enlaceEnlace copiado en el portapapeles!
| Virtual machine live migration | Host-assisted virtual machine disk cloning | Storage-assisted virtual machine disk cloning | Virtual machine snapshots | |
|---|---|---|---|---|
| OpenShift Container Storage: RBD block-mode volumes | Yes | Yes | Yes | Yes |
| OpenShift Virtualization hostpath provisioner | No | Yes | No | No |
| Other multi-node writable storage | Yes [1] | Yes | Yes [2] | Yes [2] |
| Other single-node writable storage | No | Yes | Yes [2] | Yes [2] |
- PVCs must request a ReadWriteMany access mode.
- Storage provider must support both Kubernetes and CSI snapshot APIs
You cannot live migrate virtual machines that use:
- A storage class with ReadWriteOnce (RWO) access mode
-
Passthrough features such as GPUs or SR-IOV network interfaces that have the feature gate disabled
sriovLiveMigration
Do not set the
evictionStrategy
LiveMigrate
8.18.2. Configuring local storage for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can configure local storage for your virtual machines by using the hostpath provisioner feature.
8.18.2.1. About the hostpath provisioner Copiar enlaceEnlace copiado en el portapapeles!
The hostpath provisioner is a local storage provisioner designed for OpenShift Virtualization. If you want to configure local storage for virtual machines, you must enable the hostpath provisioner first.
When you install the OpenShift Virtualization Operator, the hostpath provisioner Operator is automatically installed. To use it, you must:
Configure SELinux:
-
If you use Red Hat Enterprise Linux CoreOS (RHCOS) 8 workers, you must create a object on each node.
MachineConfig -
Otherwise, apply the SELinux label to the persistent volume (PV) backing directory on each node.
container_file_t
-
If you use Red Hat Enterprise Linux CoreOS (RHCOS) 8 workers, you must create a
-
Create a custom resource.
HostPathProvisioner -
Create a object for the hostpath provisioner.
StorageClass
The hostpath provisioner Operator deploys the provisioner as a DaemonSet on each node when you create its custom resource. In the custom resource file, you specify the backing directory for the persistent volumes that the hostpath provisioner creates.
8.18.2.2. Configuring SELinux for the hostpath provisioner on Red Hat Enterprise Linux CoreOS (RHCOS) 8 Copiar enlaceEnlace copiado en el portapapeles!
You must configure SELinux before you create the
HostPathProvisioner
MachineConfig
Prerequisites
Create a backing directory on each node for the persistent volumes (PVs) that the hostpath provisioner creates.
ImportantThe backing directory must not be located in the filesystem’s root directory because the
partition is read-only on RHCOS. For example, you can use/but not/var/<directory_name>./<directory_name>WarningIf you select a directory that shares space with your operating system, you might exhaust the space on that partition and your node might become non-functional. Create a separate partition and point the hostpath provisioner to the separate partition to avoid interference with your operating system.
Procedure
Create the
file. For example:MachineConfig$ touch machineconfig.yamlEdit the file, ensuring that you include the directory where you want the hostpath provisioner to create PVs. For example:
apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: name: 50-set-selinux-for-hostpath-provisioner labels: machineconfiguration.openshift.io/role: worker spec: config: ignition: version: 3.2.0 systemd: units: - contents: | [Unit] Description=Set SELinux chcon for hostpath provisioner Before=kubelet.service [Service] ExecStart=/usr/bin/chcon -Rt container_file_t <backing_directory_path>1 [Install] WantedBy=multi-user.target enabled: true name: hostpath-provisioner.service- 1
- Specify the backing directory where you want the provisioner to create PVs. This directory must not be located in the filesystem’s root directory (
/).
Create the
object:MachineConfig$ oc create -f machineconfig.yaml -n <namespace>
8.18.2.3. Using the hostpath provisioner to enable local storage Copiar enlaceEnlace copiado en el portapapeles!
To deploy the hostpath provisioner and enable your virtual machines to use local storage, first create a
HostPathProvisioner
Prerequisites
Create a backing directory on each node for the persistent volumes (PVs) that the hostpath provisioner creates.
ImportantThe backing directory must not be located in the filesystem’s root directory because the
partition is read-only on Red Hat Enterprise Linux CoreOS (RHCOS). For example, you can use/but not/var/<directory_name>./<directory_name>WarningIf you select a directory that shares space with your operating system, you might exhaust the space on that partition and your node becomes non-functional. Create a separate partition and point the hostpath provisioner to the separate partition to avoid interference with your operating system.
Apply the SELinux context
to the PV backing directory on each node. For example:container_file_t$ sudo chcon -t container_file_t -R <backing_directory_path>NoteIf you use Red Hat Enterprise Linux CoreOS (RHCOS) 8 workers, you must configure SELinux by using a
manifest instead.MachineConfig
Procedure
Create the
custom resource file. For example:HostPathProvisioner$ touch hostpathprovisioner_cr.yamlEdit the file, ensuring that the
value is the directory where you want the hostpath provisioner to create PVs. For example:spec.pathConfig.pathapiVersion: hostpathprovisioner.kubevirt.io/v1beta1 kind: HostPathProvisioner metadata: name: hostpath-provisioner spec: imagePullPolicy: IfNotPresent pathConfig: path: "<backing_directory_path>"1 useNamingPrefix: false2 workload:3 - 1
- Specify the backing directory where you want the provisioner to create PVs. This directory must not be located in the filesystem’s root directory (
/). - 2
- Change this value to
trueif you want to use the name of the persistent volume claim (PVC) that is bound to the created PV as the prefix of the directory name. - 3
- Optional: You can use the
spec.workloadfield to configure node placement rules for the hostpath provisioner.
NoteIf you did not create the backing directory, the provisioner attempts to create it for you. If you did not apply the
SELinux context, this can causecontainer_file_terrors.Permission deniedCreate the custom resource in the
namespace:openshift-cnv$ oc create -f hostpathprovisioner_cr.yaml -n openshift-cnv
8.18.2.4. Creating a storage class Copiar enlaceEnlace copiado en el portapapeles!
When you create a storage class, you set parameters that affect the dynamic provisioning of persistent volumes (PVs) that belong to that storage class. You cannot update a
StorageClass
When using OpenShift Virtualization with OpenShift Container Platform Container Storage, specify RBD block mode persistent volume claims (PVCs) when creating virtual machine disks. With virtual machine disks, RBD block mode volumes are more efficient and provide better performance than Ceph FS or RBD filesystem-mode PVCs.
To specify RBD block mode PVCs, use the 'ocs-storagecluster-ceph-rbd' storage class and
VolumeMode: Block
Procedure
Create a YAML file for defining the storage class. For example:
$ touch storageclass.yamlEdit the file. For example:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: hostpath-provisioner1 provisioner: kubevirt.io/hostpath-provisioner reclaimPolicy: Delete2 volumeBindingMode: WaitForFirstConsumer3 - 1
- You can optionally rename the storage class by changing this value.
- 2
- The two possible
reclaimPolicyvalues areDeleteandRetain. If you do not specify a value, the storage class defaults toDelete. - 3
- The
volumeBindingModevalue determines when dynamic provisioning and volume binding occur. SpecifyWaitForFirstConsumerto delay the binding and provisioning of a PV until after a pod that uses the persistent volume claim (PVC) is created. This ensures that the PV meets the pod’s scheduling requirements.
NoteVirtual machines use data volumes that are based on local PVs. Local PVs are bound to specific nodes. While the disk image is prepared for consumption by the virtual machine, it is possible that the virtual machine cannot be scheduled to the node where the local storage PV was previously pinned.
To solve this problem, use the Kubernetes pod scheduler to bind the PVC to a PV on the correct node. By using
withStorageClassset tovolumeBindingMode, the binding and provisioning of the PV is delayed until aWaitForFirstConsumeris created using the PVC.PodCreate the
object:StorageClass$ oc create -f storageclass.yaml
8.18.3. Creating data volumes using profiles Copiar enlaceEnlace copiado en el portapapeles!
When you create a data volume, the Containerized Data Importer (CDI) creates a persistent volume claim (PVC) and populates the PVC with your data. You can create a data volume as either a standalone resource or by using a
dataVolumeTemplate
When using OpenShift Virtualization with OpenShift Container Platform Container Storage, specify RBD block mode persistent volume claims (PVCs) when creating virtual machine disks. With virtual machine disks, RBD block mode volumes are more efficient and provide better performance than Ceph FS or RBD filesystem-mode PVCs.
To specify RBD block mode PVCs, use the 'ocs-storagecluster-ceph-rbd' storage class and
VolumeMode: Block
Whenever possible, use the storage API to optimize space allocation and maximize performance.
A storage profile is a custom resource that the CDI manages. It provides recommended storage settings based on the associated storage class. A storage profile is allocated for each storage class.
Storage profiles enable you to create data volumes quickly while reducing coding and minimizing potential errors.
For recognized storage types, the CDI provides values that optimize the creation of PVCs. However, you can configure automatic settings for a storage class if you customize the storage profile.
8.18.3.1. Creating data volumes using the storage API Copiar enlaceEnlace copiado en el portapapeles!
When you create a data volume using the storage API, the Containerized Data Interface (CDI) optimizes your persistent volume claim (PVC) allocation based on the type of storage supported by your selected storage class. You only have to specify the data volume name, namespace, and the amount of storage that you want to allocate.
For example:
-
When using Ceph RBD, is automatically set to
accessModes, which enables live migration.ReadWriteManyis set tovolumeModeto maximize performance.Block -
When you are using , more space will automatically be requested by the CDI, if required to accommodate file system overhead.
volumeMode: Filesystem
In the following YAML, using the storage API requests a data volume with two gigabytes of usable space. The user does not need to know the
volumeMode
accessModes
volumeMode
Example DataVolume definition
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: <datavolume>
spec:
source:
pvc:
namespace: "<source_namespace>"
name: "<my_vm_disk>"
storage:
resources:
requests:
storage: 2Gi
storageClassName: <storage_class>
- 1
- The name of the new data volume.
- 2
- Indicate that the source of the import is an existing persistent volume claim (PVC).
- 3
- The namespace where the source PVC exists.
- 4
- The name of the source PVC.
- 5
- Indicates allocation using the storage API.
- 6
- Specifies the amount of available space that you request for the PVC.
- 7
- Optional: The name of the storage class. If the storage class is not specified, the system default storage class is used.
8.18.3.2. Creating data volumes using the PVC API Copiar enlaceEnlace copiado en el portapapeles!
When you create a data volume using the PVC API, the Containerized Data Interface (CDI) creates the data volume based on what you specify for the following fields:
-
(
accessModes,ReadWriteOnce, orReadWriteMany)ReadOnlyMany -
(
volumeModeorFilesystem)Block -
of
capacity(storage, for example)5Gi
In the following YAML, using the PVC API allocates a data volume with a storage capacity of two gigabytes. You specify an access mode of
ReadWriteMany
Block
Filesystem
Example DataVolume definition
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: <datavolume>
spec:
source:
pvc:
namespace: "<source_namespace>"
name: "<my_vm_disk>"
pvc:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
volumeMode: Block
storageClassName: <storage_class>
- 1
- The name of the new data volume.
- 2
- In the
sourcesection,pvcindicates that the source of the import is an existing persistent volume claim (PVC). - 3
- The namespace where the source PVC exists.
- 4
- The name of the source PVC.
- 5
- Indicates allocation using the PVC API.
- 6
accessModesis required when using the PVC API.- 7
- Specifies the amount of space you are requesting for your data volume.
- 8
- Specifies that the destination is a block PVC.
- 9
- Optionally, specify the storage class. If the storage class is not specified, the system default storage class is used.
When you explicitly allocate a data volume by using the PVC API and you are not using
volumeMode: Block
File system overhead is the amount of space required by the file system to maintain its metadata. The amount of space required for file system metadata is file system dependent. Failing to account for file system overhead in your storage capacity request can result in an underlying persistent volume claim (PVC) that is not large enough to accommodate your virtual machine disk.
If you use the storage API, the CDI will factor in file system overhead and request a larger persistent volume claim (PVC) to ensure that your allocation request is successful.
8.18.3.3. Customizing the storage profile Copiar enlaceEnlace copiado en el portapapeles!
You can specify default parameters by editing the
StorageProfile
DataVolume
Prerequisites
- Ensure that your planned configuration is supported by the storage class and its provider. Specifying an incompatible configuration in a storage profile causes volume provisioning to fail.
An empty
status
If you create a data volume and omit YAML attributes and these attributes are not defined in the storage profile, then the requested storage will not be allocated and the underlying persistent volume claim (PVC) will not be created.
Procedure
Edit the storage profile. In this example, the provisioner is not recognized by CDI:
$ oc edit -n openshift-cnv storageprofile <storage_class>Example storage profile
apiVersion: cdi.kubevirt.io/v1beta1 kind: StorageProfile metadata: name: <some_unknown_provisioner_class> # ... spec: {} status: provisioner: <some_unknown_provisioner> storageClass: <some_unknown_provisioner_class>Provide the needed attribute values in the storage profile:
Example storage profile
apiVersion: cdi.kubevirt.io/v1beta1 kind: StorageProfile metadata: name: <some_unknown_provisioner_class> # ... spec: claimPropertySets: - accessModes: - ReadWriteOnce1 volumeMode: Filesystem2 status: provisioner: <some_unknown_provisioner> storageClass: <some_unknown_provisioner_class>After you save your changes, the selected values appear in the storage profile
element.status
8.18.4. Reserving PVC space for file system overhead Copiar enlaceEnlace copiado en el portapapeles!
By default, the Containerized Data Importer (CDI) reserves space for file system overhead data in persistent volume claims (PVCs) that use the
Filesystem
8.18.4.1. How file system overhead affects space for virtual machine disks Copiar enlaceEnlace copiado en el portapapeles!
When you add a virtual machine disk to a persistent volume claim (PVC) that uses the
Filesystem
- The virtual machine disk.
- The space that the Containerized Data Importer (CDI) reserves for file system overhead, such as metadata.
By default, CDI reserves 5.5% of the PVC space for overhead, reducing the space available for virtual machine disks by that amount.
If a different value works better for your use case, you can configure the overhead value by editing the
CDI
8.18.4.2. Overriding the default file system overhead value Copiar enlaceEnlace copiado en el portapapeles!
Change the amount of persistent volume claim (PVC) space that the Containerized Data Importer (CDI) reserves for file system overhead by editing the
spec.config.filesystemOverhead
CDI
Prerequisites
-
Install the OpenShift CLI ().
oc
Procedure
Open the
object for editing by running the following command:CDI$ oc edit cdiEdit the
fields, populating them with your chosen values:spec.config.filesystemOverhead... spec: config: filesystemOverhead: global: "<new_global_value>"1 storageClass: <storage_class_name>: "<new_value_for_this_storage_class>"2 - 1
- The file system overhead percentage that CDI uses across the cluster. For example,
global: "0.07"reserves 7% of the PVC for file system overhead. - 2
- The file system overhead percentage for the specified storage class. For example,
mystorageclass: "0.04"changes the default overhead value for PVCs in themystorageclassstorage class to 4%.
-
Save and exit the editor to update the object.
CDI
Verification
View the
status and verify your changes by running the following command:CDI$ oc get cdi -o yaml
8.18.5. Configuring CDI to work with namespaces that have a compute resource quota Copiar enlaceEnlace copiado en el portapapeles!
You can use the Containerized Data Importer (CDI) to import, upload, and clone virtual machine disks into namespaces that are subject to CPU and memory resource restrictions.
8.18.5.1. About CPU and memory quotas in a namespace Copiar enlaceEnlace copiado en el portapapeles!
A resource quota, defined by the
ResourceQuota
The
HyperConverged
0
8.18.5.2. Overriding CPU and memory defaults Copiar enlaceEnlace copiado en el portapapeles!
Modify the default settings for CPU and memory requests and limits for your use case by adding the
spec.resourceRequirements.storageWorkloads
HyperConverged
Prerequisites
-
Install the OpenShift CLI ().
oc
Procedure
Edit the
CR by running the following command:HyperConverged$ oc edit hco -n openshift-cnv kubevirt-hyperconvergedAdd the
stanza to the CR, setting the values based on your use case. For example:spec.resourceRequirements.storageWorkloadsapiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged spec: resourceRequirements: storageWorkloads: limits: cpu: "500m" memory: "2Gi" requests: cpu: "250m" memory: "1Gi"-
Save and exit the editor to update the CR.
HyperConverged
8.18.6. Managing data volume annotations Copiar enlaceEnlace copiado en el portapapeles!
Data volume (DV) annotations allow you to manage pod behavior. You can add one or more annotations to a data volume, which then propagates to the created importer pods.
8.18.6.1. Example: Data volume annotations Copiar enlaceEnlace copiado en el portapapeles!
This example shows how you can configure data volume (DV) annotations to control which network the importer pod uses. The
v1.multus-cni.io/default-network: bridge-network
bridge-network
k8s.v1.cni.cncf.io/networks: <network_name>
Multus network annotation example
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: dv-ann
annotations:
v1.multus-cni.io/default-network: bridge-network
spec:
source:
http:
url: "example.exampleurl.com"
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- 1
- Multus network annotation
8.18.7. Using preallocation for data volumes Copiar enlaceEnlace copiado en el portapapeles!
The Containerized Data Importer can preallocate disk space to improve write performance when creating data volumes.
You can enable preallocation for specific data volumes.
8.18.7.1. About preallocation Copiar enlaceEnlace copiado en el portapapeles!
The Containerized Data Importer (CDI) can use the QEMU preallocate mode for data volumes to improve write performance. You can use preallocation mode for importing and uploading operations and when creating blank data volumes.
If preallocation is enabled, CDI uses the better preallocation method depending on the underlying file system and device type:
fallocate-
If the file system supports it, CDI uses the operating system’s
fallocatecall to preallocate space by using theposix_fallocatefunction, which allocates blocks and marks them as uninitialized. full-
If
fallocatemode cannot be used,fullmode allocates space for the image by writing data to the underlying storage. Depending on the storage location, all the empty allocated space might be zeroed.
8.18.7.2. Enabling preallocation for a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can enable preallocation for specific data volumes by including the
spec.preallocation
oc
Preallocation mode is supported for all CDI source types.
Procedure
Specify the
field in the data volume manifest:spec.preallocationapiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: preallocated-datavolume spec: source:1 ... pvc: ... preallocation: true2
8.18.8. Uploading local disk images by using the web console Copiar enlaceEnlace copiado en el portapapeles!
You can upload a locally stored disk image file by using the web console.
8.18.8.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
- You must have a virtual machine image file in IMG, ISO, or QCOW2 format.
- If you require scratch space according to the CDI supported operations matrix, you must first define a storage class or prepare CDI scratch space for this operation to complete successfully.
8.18.8.2. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.18.8.3. Uploading an image file using the web console Copiar enlaceEnlace copiado en el portapapeles!
Use the web console to upload an image file to a new persistent volume claim (PVC). You can later use this PVC to attach the image to new virtual machines.
Prerequisites
You must have one of the following:
- A raw virtual machine image file in either ISO or IMG format.
- A virtual machine image file in QCOW2 format.
For best results, compress your image file according to the following guidelines before you upload it:
Compress a raw image file by using
orxz.gzipNoteUsing a compressed raw image file results in the most efficient upload.
Compress a QCOW2 image file by using the method that is recommended for your client:
- If you use a Linux client, sparsify the QCOW2 file by using the virt-sparsify tool.
-
If you use a Windows client, compress the QCOW2 file by using or
xz.gzip
Procedure
- From the side menu of the web console, click Storage → Persistent Volume Claims.
- Click the Create Persistent Volume Claim drop-down list to expand it.
- Click With Data Upload Form to open the Upload Data to Persistent Volume Claim page.
- Click Browse to open the file manager and select the image that you want to upload, or drag the file into the Drag a file here or browse to upload field.
Optional: Set this image as the default image for a specific operating system.
- Select the Attach this data to a virtual machine operating system check box.
- Select an operating system from the list.
- The Persistent Volume Claim Name field is automatically filled with a unique name and cannot be edited. Take note of the name assigned to the PVC so that you can identify it later, if necessary.
- Select a storage class from the Storage Class list.
In the Size field, enter the size value for the PVC. Select the corresponding unit of measurement from the drop-down list.
WarningThe PVC size must be larger than the size of the uncompressed virtual disk.
- Select an Access Mode that matches the storage class that you selected.
- Click Upload.
8.18.9. Uploading local disk images by using the virtctl tool Copiar enlaceEnlace copiado en el portapapeles!
You can upload a locally stored disk image to a new or existing data volume by using the
virtctl
8.18.9.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
Install the package.
kubevirt-virtctl - If you require scratch space according to the CDI supported operations matrix, you must first define a storage class or prepare CDI scratch space for this operation to complete successfully.
8.18.9.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.18.9.3. Creating an upload data volume Copiar enlaceEnlace copiado en el portapapeles!
You can manually create a data volume with an
upload
Procedure
Create a data volume configuration that specifies
:spec: source: upload{}apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <upload-datavolume>1 spec: source: upload: {} pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi>2 Create the data volume by running the following command:
$ oc create -f <upload-datavolume>.yaml
8.18.9.4. Uploading a local disk image to a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can use the
virtctl
After you upload a local disk image, you can add it to a virtual machine.
Prerequisites
You must have one of the following:
- A raw virtual machine image file in either ISO or IMG format.
- A virtual machine image file in QCOW2 format.
For best results, compress your image file according to the following guidelines before you upload it:
Compress a raw image file by using
orxz.gzipNoteUsing a compressed raw image file results in the most efficient upload.
Compress a QCOW2 image file by using the method that is recommended for your client:
- If you use a Linux client, sparsify the QCOW2 file by using the virt-sparsify tool.
-
If you use a Windows client, compress the QCOW2 file by using or
xz.gzip
-
The package must be installed on the client machine.
kubevirt-virtctl - The client machine must be configured to trust the OpenShift Container Platform router’s certificate.
Procedure
Identify the following items:
- The name of the upload data volume that you want to use. If this data volume does not exist, it is created automatically.
- The size of the data volume, if you want it to be created during the upload procedure. The size must be greater than or equal to the size of the disk image.
- The file location of the virtual machine disk image that you want to upload.
Upload the disk image by running the
command. Specify the parameters that you identified in the previous step. For example:virtctl image-upload$ virtctl image-upload dv <datavolume_name> \1 --size=<datavolume_size> \2 --image-path=</path/to/image> \3 Note-
If you do not want to create a new data volume, omit the parameter and include the
--sizeflag.--no-create - When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk.
-
To allow insecure server connections when using HTTPS, use the parameter. Be aware that when you use the
--insecureflag, the authenticity of the upload endpoint is not verified.--insecure
-
If you do not want to create a new data volume, omit the
Optional. To verify that a data volume was created, view all data volumes by running the following command:
$ oc get dvs
8.18.9.5. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.18.10. Uploading a local disk image to a block storage data volume Copiar enlaceEnlace copiado en el portapapeles!
You can upload a local disk image into a block data volume by using the
virtctl
In this workflow, you create a local block device to use as a persistent volume, associate this block volume with an
upload
virtctl
8.18.10.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
Install the package.
kubevirt-virtctl - If you require scratch space according to the CDI supported operations matrix, you must first define a storage class or prepare CDI scratch space for this operation to complete successfully.
8.18.10.2. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.18.10.3. About block persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
A block persistent volume (PV) is a PV that is backed by a raw block device. These volumes do not have a file system and can provide performance benefits for virtual machines by reducing overhead.
Raw block volumes are provisioned by specifying
volumeMode: Block
8.18.10.4. Creating a local block persistent volume Copiar enlaceEnlace copiado en el portapapeles!
Create a local block persistent volume (PV) on a node by populating a file and mounting it as a loop device. You can then reference this loop device in a PV manifest as a
Block
Procedure
-
Log in as to the node on which to create the local PV. This procedure uses
rootfor its examples.node01 Create a file and populate it with null characters so that it can be used as a block device. The following example creates a file
with a size of 2Gb (20 100Mb blocks):loop10$ dd if=/dev/zero of=<loop10> bs=100M count=20Mount the
file as a loop device.loop10$ losetup </dev/loop10>d3 <loop10>1 2 Create a
manifest that references the mounted loop device.PersistentVolumekind: PersistentVolume apiVersion: v1 metadata: name: <local-block-pv10> annotations: spec: local: path: </dev/loop10>1 capacity: storage: <2Gi> volumeMode: Block2 storageClassName: local3 accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - <node01>4 Create the block PV.
# oc create -f <local-block-pv10.yaml>1 - 1
- The file name of the persistent volume created in the previous step.
8.18.10.5. Creating an upload data volume Copiar enlaceEnlace copiado en el portapapeles!
You can manually create a data volume with an
upload
Procedure
Create a data volume configuration that specifies
:spec: source: upload{}apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <upload-datavolume>1 spec: source: upload: {} pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi>2 Create the data volume by running the following command:
$ oc create -f <upload-datavolume>.yaml
8.18.10.6. Uploading a local disk image to a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can use the
virtctl
After you upload a local disk image, you can add it to a virtual machine.
Prerequisites
You must have one of the following:
- A raw virtual machine image file in either ISO or IMG format.
- A virtual machine image file in QCOW2 format.
For best results, compress your image file according to the following guidelines before you upload it:
Compress a raw image file by using
orxz.gzipNoteUsing a compressed raw image file results in the most efficient upload.
Compress a QCOW2 image file by using the method that is recommended for your client:
- If you use a Linux client, sparsify the QCOW2 file by using the virt-sparsify tool.
-
If you use a Windows client, compress the QCOW2 file by using or
xz.gzip
-
The package must be installed on the client machine.
kubevirt-virtctl - The client machine must be configured to trust the OpenShift Container Platform router’s certificate.
Procedure
Identify the following items:
- The name of the upload data volume that you want to use. If this data volume does not exist, it is created automatically.
- The size of the data volume, if you want it to be created during the upload procedure. The size must be greater than or equal to the size of the disk image.
- The file location of the virtual machine disk image that you want to upload.
Upload the disk image by running the
command. Specify the parameters that you identified in the previous step. For example:virtctl image-upload$ virtctl image-upload dv <datavolume_name> \1 --size=<datavolume_size> \2 --image-path=</path/to/image> \3 Note-
If you do not want to create a new data volume, omit the parameter and include the
--sizeflag.--no-create - When uploading a disk image to a PVC, the PVC size must be larger than the size of the uncompressed virtual disk.
-
To allow insecure server connections when using HTTPS, use the parameter. Be aware that when you use the
--insecureflag, the authenticity of the upload endpoint is not verified.--insecure
-
If you do not want to create a new data volume, omit the
Optional. To verify that a data volume was created, view all data volumes by running the following command:
$ oc get dvs
8.18.10.7. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.18.11. Managing offline virtual machine snapshots Copiar enlaceEnlace copiado en el portapapeles!
You can create, restore, and delete virtual machine (VM) snapshots for VMs that are powered off (offline). OpenShift Virtualization supports offline VM snapshots on:
- Red Hat OpenShift Container Storage
- Any other storage provider with the Container Storage Interface (CSI) driver that supports the Kubernetes Volume Snapshot API
8.18.11.1. About virtual machine snapshots Copiar enlaceEnlace copiado en el portapapeles!
A snapshot represents the state and data of a virtual machine (VM) at a specific point in time. You can use a snapshot to restore an existing VM to a previous state (represented by the snapshot) for backup and disaster recovery or to rapidly roll back to a previous development version.
An offline VM snapshot is created from a VM that is powered off (Stopped state). The snapshot stores a copy of each Container Storage Interface (CSI) volume attached to the VM and a copy of the VM specification and metadata. Snapshots cannot be changed after creation.
With the offline VM snapshots feature, cluster administrators and application developers can:
- Create a new snapshot
- List all snapshots attached to a specific VM
- Restore a VM from a snapshot
- Delete an existing VM snapshot
8.18.11.1.1. Virtual machine snapshot controller and custom resource definitions (CRDs) Copiar enlaceEnlace copiado en el portapapeles!
The VM snapshot feature introduces three new API objects defined as CRDs for managing snapshots:
-
: Represents a user request to create a snapshot. It contains information about the current state of the VM.
VirtualMachineSnapshot -
: Represents a provisioned resource on the cluster (a snapshot). It is created by the VM snapshot controller and contains references to all resources required to restore the VM.
VirtualMachineSnapshotContent -
: Represents a user request to restore a VM from a snapshot.
VirtualMachineRestore
The VM snapshot controller binds a
VirtualMachineSnapshotContent
VirtualMachineSnapshot
8.18.11.2. Creating an offline virtual machine snapshot in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can create a virtual machine (VM) snapshot by using the web console.
The VM snapshot only includes disks that meet the following requirements:
- Must be either a data volume or persistent volume claim
- Belong to a storage class that supports Container Storage Interface (CSI) volume snapshots
If your VM storage includes disks that do not support snapshots, you can either edit them or contact your cluster administrator.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- If the virtual machine is running, click Actions → Stop Virtual Machine to power it down.
- Click the Snapshots tab and then click Take Snapshot.
- Fill in the Snapshot Name and optional Description fields.
- Expand Disks included in this Snapshot to see the storage volumes to be included in the snapshot.
- If your VM has disks that cannot be included in the snapshot and you still wish to proceed, select the I am aware of this warning and wish to proceed checkbox.
- Click Save.
8.18.11.3. Creating an offline virtual machine snapshot in the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can create a virtual machine (VM) snapshot for an offline VM by creating a
VirtualMachineSnapshot
Prerequisites
- Ensure that the persistent volume claims (PVCs) are in a storage class that supports Container Storage Interface (CSI) volume snapshots.
-
Install the OpenShift CLI ().
oc - Power down the VM for which you want to create a snapshot.
Procedure
Create a YAML file to define a
object that specifies the name of the newVirtualMachineSnapshotand the name of the source VM.VirtualMachineSnapshotFor example:
apiVersion: snapshot.kubevirt.io/v1alpha1 kind: VirtualMachineSnapshot metadata: name: my-vmsnapshot1 spec: source: apiGroup: kubevirt.io kind: VirtualMachine name: my-vm2 Create the
resource. The snapshot controller creates aVirtualMachineSnapshotobject, binds it to theVirtualMachineSnapshotContentand updates theVirtualMachineSnapshotandstatusfields of thereadyToUseobject.VirtualMachineSnapshot$ oc create -f <my-vmsnapshot>.yaml
Verification
Verify that the
object is created and bound withVirtualMachineSnapshot. TheVirtualMachineSnapshotContentflag must be set toreadyToUse.true$ oc describe vmsnapshot <my-vmsnapshot>Example output
apiVersion: snapshot.kubevirt.io/v1alpha1 kind: VirtualMachineSnapshot metadata: creationTimestamp: "2020-09-30T14:41:51Z" finalizers: - snapshot.kubevirt.io/vmsnapshot-protection generation: 5 name: mysnap namespace: default resourceVersion: "3897" selfLink: /apis/snapshot.kubevirt.io/v1alpha1/namespaces/default/virtualmachinesnapshots/my-vmsnapshot uid: 28eedf08-5d6a-42c1-969c-2eda58e2a78d spec: source: apiGroup: kubevirt.io kind: VirtualMachine name: my-vm status: conditions: - lastProbeTime: null lastTransitionTime: "2020-09-30T14:42:03Z" reason: Operation complete status: "False"1 type: Progressing - lastProbeTime: null lastTransitionTime: "2020-09-30T14:42:03Z" reason: Operation complete status: "True"2 type: Ready creationTime: "2020-09-30T14:42:03Z" readyToUse: true3 sourceUID: 355897f3-73a0-4ec4-83d3-3c2df9486f4f virtualMachineSnapshotContentName: vmsnapshot-content-28eedf08-5d6a-42c1-969c-2eda58e2a78d4 - 1
- The
statusfield of theProgressingcondition specifies if the snapshot is still being created. - 2
- The
statusfield of theReadycondition specifies if the snapshot creation process is complete. - 3
- Specifies if the snapshot is ready to be used.
- 4
- Specifies that the snapshot is bound to a
VirtualMachineSnapshotContentobject created by the snapshot controller.
-
Check the property of the
spec:volumeBackupsresource to verify that the expected PVCs are included in the snapshot.VirtualMachineSnapshotContent
8.18.11.4. Restoring a virtual machine from a snapshot in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can restore a virtual machine (VM) to a previous configuration represented by a snapshot in the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- If the virtual machine is running, click Actions → Stop Virtual Machine to power it down.
- Click the Snapshots tab. The page displays a list of snapshots associated with the virtual machine.
Choose one of the following methods to restore a VM snapshot:
- For the snapshot that you want to use as the source to restore the VM, click Restore.
- Select a snapshot to open the Snapshot Details screen and click Actions → Restore Virtual Machine Snapshot.
- In the confirmation pop-up window, click Restore to restore the VM to its previous configuration represented by the snapshot.
8.18.11.5. Restoring a virtual machine from a snapshot in the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can restore an existing virtual machine (VM) to a previous configuration by using a VM snapshot .
Prerequisites
-
Install the OpenShift CLI ().
oc - Power down the VM you want to restore to a previous state.
Procedure
Create a YAML file to define a
object that specifies the name of the VM you want to restore and the name of the snapshot to be used as the source.VirtualMachineRestoreFor example:
apiVersion: snapshot.kubevirt.io/v1alpha1 kind: VirtualMachineRestore metadata: name: my-vmrestore1 spec: target: apiGroup: kubevirt.io kind: VirtualMachine name: my-vm2 virtualMachineSnapshotName: my-vmsnapshot3 Create the
resource. The snapshot controller updates the status fields of theVirtualMachineRestoreobject and replaces the existing VM configuration with the snapshot content.VirtualMachineRestore$ oc create -f <my-vmrestore>.yaml
Verification
Verify that the VM is restored to the previous state represented by the snapshot. The
flag must be set tocomplete.true$ oc get vmrestore <my-vmrestore>Example output
apiVersion: snapshot.kubevirt.io/v1alpha1 kind: VirtualMachineRestore metadata: creationTimestamp: "2020-09-30T14:46:27Z" generation: 5 name: my-vmrestore namespace: default ownerReferences: - apiVersion: kubevirt.io/v1 blockOwnerDeletion: true controller: true kind: VirtualMachine name: my-vm uid: 355897f3-73a0-4ec4-83d3-3c2df9486f4f resourceVersion: "5512" selfLink: /apis/snapshot.kubevirt.io/v1alpha1/namespaces/default/virtualmachinerestores/my-vmrestore uid: 71c679a8-136e-46b0-b9b5-f57175a6a041 spec: target: apiGroup: kubevirt.io kind: VirtualMachine name: my-vm virtualMachineSnapshotName: my-vmsnapshot status: complete: true1 conditions: - lastProbeTime: null lastTransitionTime: "2020-09-30T14:46:28Z" reason: Operation complete status: "False"2 type: Progressing - lastProbeTime: null lastTransitionTime: "2020-09-30T14:46:28Z" reason: Operation complete status: "True"3 type: Ready deletedDataVolumes: - test-dv1 restoreTime: "2020-09-30T14:46:28Z" restores: - dataVolumeName: restore-71c679a8-136e-46b0-b9b5-f57175a6a041-datavolumedisk1 persistentVolumeClaim: restore-71c679a8-136e-46b0-b9b5-f57175a6a041-datavolumedisk1 volumeName: datavolumedisk1 volumeSnapshotName: vmsnapshot-28eedf08-5d6a-42c1-969c-2eda58e2a78d-volume-datavolumedisk1
8.18.11.6. Deleting a virtual machine snapshot in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can delete an existing virtual machine snapshot by using the web console.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click the Snapshots tab. The page displays a list of snapshots associated with the virtual machine.
Choose one of the following methods to delete a virtual machine snapshot:
-
Click the Options menu
of the virtual machine snapshot that you want to delete and select Delete Virtual Machine Snapshot.
- Select a snapshot to open the Snapshot Details screen and click Actions → Delete Virtual Machine Snapshot.
-
Click the Options menu
- In the confirmation pop-up window, click Delete to delete the snapshot.
8.18.11.7. Deleting a virtual machine snapshot in the CLI Copiar enlaceEnlace copiado en el portapapeles!
You can delete an existing virtual machine (VM) snapshot by deleting the appropriate
VirtualMachineSnapshot
Prerequisites
-
Install the OpenShift CLI ().
oc
Procedure
Delete the
object. The snapshot controller deletes theVirtualMachineSnapshotalong with the associatedVirtualMachineSnapshotobject.VirtualMachineSnapshotContent$ oc delete vmsnapshot <my-vmsnapshot>
Verification
Verify that the snapshot is deleted and no longer attached to this VM:
$ oc get vmsnapshot
8.18.12. Moving a local virtual machine disk to a different node Copiar enlaceEnlace copiado en el portapapeles!
Virtual machines that use local volume storage can be moved so that they run on a specific node.
You might want to move the virtual machine to a specific node for the following reasons:
- The current node has limitations to the local storage configuration.
- The new node is better optimized for the workload of that virtual machine.
To move a virtual machine that uses local storage, you must clone the underlying volume by using a data volume. After the cloning operation is complete, you can edit the virtual machine configuration so that it uses the new data volume, or add the new data volume to another virtual machine.
When you enable preallocation globally, or for a single data volume, the Containerized Data Importer (CDI) preallocates disk space during cloning. Preallocation enhances write performance. For more information, see Using preallocation for data volumes.
Users without the
cluster-admin
8.18.12.1. Cloning a local volume to another node Copiar enlaceEnlace copiado en el portapapeles!
You can move a virtual machine disk so that it runs on a specific node by cloning the underlying persistent volume claim (PVC).
To ensure the virtual machine disk is cloned to the correct node, you must either create a new persistent volume (PV) or identify one on the correct node. Apply a unique label to the PV so that it can be referenced by the data volume.
The destination PV must be the same size or larger than the source PVC. If the destination PV is smaller than the source PVC, the cloning operation fails.
Prerequisites
- The virtual machine must not be running. Power down the virtual machine before cloning the virtual machine disk.
Procedure
Either create a new local PV on the node, or identify a local PV already on the node:
Create a local PV that includes the
parameters. The following manifest creates anodeAffinity.nodeSelectorTermslocal PV on10Gi.node01kind: PersistentVolume apiVersion: v1 metadata: name: <destination-pv>1 annotations: spec: accessModes: - ReadWriteOnce capacity: storage: 10Gi2 local: path: /mnt/local-storage/local/disk13 nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - node014 persistentVolumeReclaimPolicy: Delete storageClassName: local volumeMode: FilesystemIdentify a PV that already exists on the target node. You can identify the node where a PV is provisioned by viewing the
field in its configuration:nodeAffinity$ oc get pv <destination-pv> -o yamlThe following snippet shows that the PV is on
:node01Example output
... spec: nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname1 operator: In values: - node012 ...
Add a unique label to the PV:
$ oc label pv <destination-pv> node=node01Create a data volume manifest that references the following:
- The PVC name and namespace of the virtual machine.
- The label you applied to the PV in the previous step.
The size of the destination PV.
apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <clone-datavolume>1 spec: source: pvc: name: "<source-vm-disk>"2 namespace: "<source-namespace>"3 pvc: accessModes: - ReadWriteOnce selector: matchLabels: node: node014 resources: requests: storage: <10Gi>5 - 1
- The name of the new data volume.
- 2
- The name of the source PVC. If you do not know the PVC name, you can find it in the virtual machine configuration:
spec.volumes.persistentVolumeClaim.claimName. - 3
- The namespace where the source PVC exists.
- 4
- The label that you applied to the PV in the previous step.
- 5
- The size of the destination PV.
Start the cloning operation by applying the data volume manifest to your cluster:
$ oc apply -f <clone-datavolume.yaml>
The data volume clones the PVC of the virtual machine into the PV on the specific node.
8.18.13. Expanding virtual storage by adding blank disk images Copiar enlaceEnlace copiado en el portapapeles!
You can increase your storage capacity or create new data partitions by adding blank disk images to OpenShift Virtualization.
8.18.13.1. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.18.13.2. Creating a blank disk image with data volumes Copiar enlaceEnlace copiado en el portapapeles!
You can create a new blank disk image in a persistent volume claim by customizing and deploying a data volume configuration file.
Prerequisites
- At least one available persistent volume.
-
Install the OpenShift CLI ().
oc
Procedure
Edit the data volume configuration file:
apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: blank-image-datavolume spec: source: blank: {} pvc: # Optional: Set the storage class or omit to accept the default # storageClassName: "hostpath" accessModes: - ReadWriteOnce resources: requests: storage: 500MiCreate the blank disk image by running the following command:
$ oc create -f <blank-image-datavolume>.yaml
8.18.13.3. Template: Data volume configuration file for blank disk images Copiar enlaceEnlace copiado en el portapapeles!
blank-image-datavolume.yaml
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: blank-image-datavolume
spec:
source:
blank: {}
pvc:
# Optional: Set the storage class or omit to accept the default
# storageClassName: "hostpath"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
8.18.14. Cloning a data volume using smart-cloning Copiar enlaceEnlace copiado en el portapapeles!
Smart-cloning is a built-in feature of OpenShift Container Platform Storage (OCS), designed to enhance performance of the cloning process. Clones created with smart-cloning are faster and more efficient than host-assisted cloning.
You do not need to perform any action to enable smart-cloning, but you need to ensure your storage environment is compatible with smart-cloning to use this feature.
When you create a data volume with a persistent volume claim (PVC) source, you automatically initiate the cloning process. You always receive a clone of the data volume, if your environment supports smart-cloning or not. However, you will only receive the performance benefits of smart cloning if you storage provider supports smart-cloning.
8.18.14.1. Understanding smart-cloning Copiar enlaceEnlace copiado en el portapapeles!
When a data volume is smart-cloned, the following occurs:
- A snapshot of the source persistent volume claim (PVC) is created.
- A PVC is created from the snapshot.
- The snapshot is deleted.
8.18.14.2. Cloning a data volume Copiar enlaceEnlace copiado en el portapapeles!
Prerequisites
For smart-cloning to occur, the following conditions are required:
- Your storage provider must support snapshots.
- The source and target PVCs must be defined to the same storage class.
-
The object must reference the storage class defined to both the source and target PVCs.
VolumeSnapshotClass
Procedure
To initiate cloning of a data volume:
Create a YAML file for a
object that specifies the name of the new data volume and the name and namespace of the source PVC. In this example, because you specify the storage API, there is no need to specify accessModes or volumeMode. The optimal values will be calculated for you automatically.DataVolumeapiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: <cloner-datavolume>1 spec: source: pvc: namespace: "<source-namespace>"2 name: "<my-favorite-vm-disk>"3 storage:4 resources: requests: storage: <2Gi>5 Start cloning the PVC by creating the data volume:
$ oc create -f <cloner-datavolume>.yamlNoteData volumes prevent a virtual machine from starting before the PVC is prepared, so you can create a virtual machine that references the new data volume while the PVC clones.
8.18.15. Creating and using boot sources Copiar enlaceEnlace copiado en el portapapeles!
A boot source contains a bootable operating system (OS) and all of the configuration settings for the OS, such as drivers.
You use a boot source to create virtual machine templates with specific configurations. These templates can be used to create any number of available virtual machines.
Quick Start tours are available in the OpenShift Container Platform web console to assist you in creating a custom boot source, uploading a boot source, and other tasks. Select Quick Starts from the Help menu to view the Quick Start tours.
8.18.15.1. About virtual machines and boot sources Copiar enlaceEnlace copiado en el portapapeles!
Virtual machines consist of a virtual machine definition and one or more disks that are backed by data volumes. Virtual machine templates enable you to create virtual machines using predefined virtual machine specifications.
Every virtual machine template requires a boot source, which is a fully configured virtual machine disk image including configured drivers. Each virtual machine template contains a virtual machine definition with a pointer to the boot source. Each boot source has a predefined name and namespace. For some operating systems, a boot source is automatically provided. If it is not provided, then an administrator must prepare a custom boot source.
Provided boot sources are updated automatically to the latest version of the operating system. For auto-updated boot sources, persistent volume claims (PVCs) are created with the cluster’s default storage class. If you select a different default storage class after configuration, you must delete the existing data volumes in the cluster namespace that are configured with the previous default storage class.
To use the boot sources feature, install the latest release of OpenShift Virtualization. The namespace
openshift-virtualization-os-images
Define a boot source by using a persistent volume claim (PVC) that is populated by uploading a local file, cloning an existing PVC, importing from a registry, or by URL. Attach a boot source to a virtual machine template by using the web console. After the boot source is attached to a virtual machine template, you create any number of fully configured ready-to-use virtual machines from the template.
8.18.15.2. Importing a Red Hat Enterprise Linux image as a boot source Copiar enlaceEnlace copiado en el portapapeles!
You can import a Red Hat Enterprise Linux (RHEL) image as a boot source by specifying the URL address for the image.
Prerequisites
- You must have access to the web server with the operating system image. For example: Red Hat Enterprise Linux web page with images.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Identify the RHEL template for which you want to configure a boot source and click Add source.
- In the Add boot source to template window, select Import via URL (creates PVC) from the Boot source type list.
- Click RHEL download page to access the Red Hat Customer Portal. A list of available installers and images is displayed on the Download Red Hat Enterprise Linux page.
- Identify the Red Hat Enterprise Linux KVM guest image that you want to download. Right-click Download Now, and copy the URL for the image.
- In the Add boot source to template window, paste the copied URL of the guest image into the Import URL field, and click Save and import.
Verification
To verify that a boot source was added to the template:
- Click the Templates tab.
- Confirm that the tile for this template displays a green checkmark.
You can now use this template to create RHEL virtual machines.
8.18.15.3. Adding a boot source for a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
A boot source can be configured for any virtual machine template that you want to use for creating virtual machines or custom templates. When virtual machine templates are configured with a boot source, they are labeled Available in the Templates tab. After you add a boot source to a template, you can create a new virtual machine from the template.
There are four methods for selecting and adding a boot source in the web console:
- Upload local file (creates PVC)
- Import via URL (creates PVC)
- Clone existing PVC (creates PVC)
- Import via Registry (creates PVC)
Prerequisites
-
To add a boot source, you must be logged in as a user with the RBAC role or as an administrator. You do not need special privileges to create a virtual machine from a template with a boot source added.
os-images.kubevirt.io:edit - To upload a local file, the operating system image file must exist on your local machine.
- To import via URL, access to the web server with the operating system image is required. For example: the Red Hat Enterprise Linux web page with images.
- To clone an existing PVC, access to the project with a PVC is required.
- To import via registry, access to the container registry is required.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Identify the virtual machine template for which you want to configure a boot source and click Add source.
- In the Add boot source to template window, click Select boot source, select a method for creating a persistent volume claim (PVC): Upload local file, Import via URL, Clone existing PVC, or Import via Registry.
- Optional: Click This is a CD-ROM boot source to mount a CD-ROM and use it to install the operating system on to an empty disk. The additional empty disk is automatically created and mounted by OpenShift Virtualization. If the additional disk is not needed, you can remove it when you create the virtual machine.
Enter a value for Persistent Volume Claim size to specify the PVC size that is adequate for the uncompressed image and any additional space that is required.
- Optional: Enter a name for Source provider to associate the name with this template.
Optional: Advanced Storage settings: Click Storage class and select the storage class that is used to create the disk. Typically, this storage class is the default storage class that is created for use by all PVCs.
NoteProvided boot sources are updated automatically to the latest version of the operating system. For auto-updated boot sources, persistent volume claims (PVCs) are created with the cluster’s default storage class. If you select a different default storage class after configuration, you must delete the existing data volumes in the cluster namespace that are configured with the previous default storage class.
Optional: Advanced Storage settings: Click Access mode and select an access mode for the persistent volume:
- Single User (RWO) mounts the volume as read-write by a single node.
- Shared Access (RWX) mounts the volume as read-write by many nodes.
- Read Only (ROX) mounts the volume as read-only by many nodes.
- Optional: Advanced Storage settings: Click Volume mode if you want to select Block instead of the default value Filesystem. OpenShift Virtualization can statically provision raw block volumes. These volumes do not have a file system, and can provide performance benefits for applications that either write to the disk directly or implement their own storage service.
Select the appropriate method to save your boot source:
- Click Save and upload if you uploaded a local file.
- Click Save and import if you imported content from a URL or the registry.
- Click Save and clone if you cloned an existing PVC.
Your custom virtual machine template with a boot source is listed in the Templates tab, and you can create virtual machines by using this template.
8.18.15.4. Creating a virtual machine from a template with an attached boot source Copiar enlaceEnlace copiado en el portapapeles!
After you add a boot source to a template, you can create a new virtual machine from the template.
Procedure
- In the OpenShift Container Platform web console, click Workloads > Virtualization in the side menu.
- From the Virtual Machines tab or the Templates tab, click Create and select Virtual Machine with Wizard.
- In the Select a template step, select an OS from the Operating System list that has the (Source available) label next to the OS and version name. The (Source available) label indicates that a boot source is available for this OS.
- Click Review and Confirm.
- Review your virtual machine settings and edit them, if required.
- Click Create Virtual Machine to create your virtual machine. The Successfully created virtual machine page is displayed.
8.18.15.5. Creating a custom boot source Copiar enlaceEnlace copiado en el portapapeles!
You can prepare a custom disk image, based on an existing disk image, for use as a boot source.
Use this procedure to complete the following tasks:
- Preparing a custom disk image
- Creating a boot source from the custom disk image
- Attaching the boot source to a custom template
Procedure
- In the OpenShift Virtualization console, click Workloads > Virtualization from the side menu.
- Click the Templates tab.
- Click the link in the Source provider column for the template you want to customize. A window displays, indicating that the template currently has a defined source.
- In the window, click the Customize source link.
- Click Continue in the About boot source customization window to proceed with customization after reading the information provided about the boot source customization process.
On the Prepare boot source customization page, in the Define new template section:
- Select the New template namespace field and then choose a project.
- Enter the name of the custom template in the New template name field.
- Enter the name of the template provider in the New template provider field.
- Select the New template support field and then choose the appropriate value, indicating support contacts for the custom template you create.
- Select the New template flavor field and then choose the appropriate CPU and memory values for the custom image you create.
-
In the Prepare boot source for customization section, customize the YAML script, if needed, to define login credentials. Otherwise, the script generates default credentials for you.
cloud-init - Click Start Customization. The customization process begins and the Preparing boot source customization page displays, followed by the Customize boot source page. The Customize boot source page displays the output of the running script. When the script completes, your custom image is available.
- In the VNC console, click show password in the Guest login credentials section. Your login credentials display.
- When the image is ready for login, sign in with the VNC Console by providing the user name and password displayed in the Guest login credentials section.
- Verify the custom image works as expected. If it does, click Make this boot source available.
- In the Finish customization and make template available window, select I have sealed the boot source so it can be used as a template and then click Apply.
- On the Finishing boot source customization page, wait for the template creation process to complete. Click Navigate to template details or Navigate to template list to view your customized template, created from your custom boot source.
8.18.16. Hot-plugging virtual disks Copiar enlaceEnlace copiado en el portapapeles!
Hot-plugging virtual disks is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.
For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.
Hot-plug and hot-unplug virtual disks when you want to add or remove them without stopping your virtual machine or virtual machine instance. This capability is helpful when you need to add storage to a running virtual machine without incurring down-time.
When you hot-plug a virtual disk, you attach a virtual disk to a virtual machine instance while the virtual machine is running.
When you hot-unplug a virtual disk, you detach a virtual disk from a virtual machine instance while the virtual machine is running.
Only data volumes and persistent volume claims (PVCs) can be hot-plugged and hot-unplugged. You cannot hot-plug or hot-unplug container disks.
When you attach a hot-plugged virtual disk to a virtual machine, you cannot perform live migration on the virtual machine. If you attempt to perform live migration on a virtual machine with a hot-plugged disk attached, live migration fails.
8.18.16.1. Hot-plugging a virtual disk using the CLI Copiar enlaceEnlace copiado en el portapapeles!
Hot-plug virtual disks that you want to attach to a virtual machine instance (VMI) while a virtual machine is running.
Prerequisites
- You must have a running virtual machine to hot-plug a virtual disk.
- You must have at least one data volume or persistent volume claim (PVC) available for hot-plugging.
Procedure
Hot-plug a virtual disk by running the following command:
$ virtctl addvolume <virtual-machine|virtual-machine-instance> --volume-name=<datavolume|PVC> \ [--persist] [--serial=<label-name>]-
Use the optional flag to add the hot-plugged disk to the virtual machine specification as a permanently mounted virtual disk. Stop, restart, or reboot the virtual machine to permanently mount the virtual disk. After specifying the
--persistflag, you can no longer hot-plug or hot-unplug the virtual disk. The--persistflag applies to virtual machines, not virtual machine interfaces.--persist -
The optional flag allows you to add an alphanumeric string label of your choice. This helps you to identify the hot-plugged disk in a guest virtual machine. If you do not specify this option, the label defaults to the name of the hot-plugged data volume or PVC.
--serial
-
Use the optional
8.18.16.2. Hot-unplugging a virtual disk using the CLI Copiar enlaceEnlace copiado en el portapapeles!
Hot-unplug virtual disks that you want to detach from a virtual machine instance (VMI) while a virtual machine is running.
Prerequisites
- Your virtual machine must be running.
- You must have at least one data volume or persistent volume claim (PVC) available and hot-plugged.
Procedure
Hot-unplug a virtual disk by running the following command:
$ virtctl removevolume <virtual-machine|virtual-machine-instance> --volume-name=<datavolume|PVC>
8.18.17. Using container disks with virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You can build a virtual machine image into a container disk and store it in your container registry. You can then import the container disk into persistent storage for a virtual machine or attach it directly to the virtual machine for ephemeral storage.
If you use large container disks, I/O traffic might increase, impacting worker nodes. This can lead to unavailable nodes. You can resolve this by:
8.18.17.1. About container disks Copiar enlaceEnlace copiado en el portapapeles!
A container disk is a virtual machine image that is stored as a container image in a container image registry. You can use container disks to deliver the same disk images to multiple virtual machines and to create large numbers of virtual machine clones.
A container disk can either be imported into a persistent volume claim (PVC) by using a data volume that is attached to a virtual machine, or attached directly to a virtual machine as an ephemeral
containerDisk
8.18.17.1.1. Importing a container disk into a PVC by using a data volume Copiar enlaceEnlace copiado en el portapapeles!
Use the Containerized Data Importer (CDI) to import the container disk into a PVC by using a data volume. You can then attach the data volume to a virtual machine for persistent storage.
8.18.17.1.2. Attaching a container disk to a virtual machine as a containerDisk volume Copiar enlaceEnlace copiado en el portapapeles!
A
containerDisk
containerDisk
Use
containerDisk
Using
containerDisk
8.18.17.2. Preparing a container disk for virtual machines Copiar enlaceEnlace copiado en el portapapeles!
You must build a container disk with a virtual machine image and push it to a container registry before it can used with a virtual machine. You can then either import the container disk into a PVC using a data volume and attach it to a virtual machine, or you can attach the container disk directly to a virtual machine as an ephemeral
containerDisk
The size of a disk image inside a container disk is limited by the maximum layer size of the registry where the container disk is hosted.
For Red Hat Quay, you can change the maximum layer size by editing the YAML configuration file that is created when Red Hat Quay is first deployed.
Prerequisites
-
Install if it is not already installed.
podman - The virtual machine image must be either QCOW2 or RAW format.
Procedure
Create a Dockerfile to build the virtual machine image into a container image. The virtual machine image must be owned by QEMU, which has a UID of
, and placed in the107directory inside the container. Permissions for the/disk/directory must then be set to/disk/.0440The following example uses the Red Hat Universal Base Image (UBI) to handle these configuration changes in the first stage, and uses the minimal
image in the second stage to store the result:scratch$ cat > Dockerfile << EOF FROM registry.access.redhat.com/ubi8/ubi:latest AS builder ADD --chown=107:107 <vm_image>.qcow2 /disk/1 RUN chmod 0440 /disk/* FROM scratch COPY --from=builder /disk/* /disk/ EOF- 1
- Where <vm_image> is the virtual machine image in either QCOW2 or RAW format.
To use a remote virtual machine image, replace<vm_image>.qcow2with the complete url for the remote image.
Build and tag the container:
$ podman build -t <registry>/<container_disk_name>:latest .Push the container image to the registry:
$ podman push <registry>/<container_disk_name>:latest
If your container registry does not have TLS you must add it as an insecure registry before you can import container disks into persistent storage.
8.18.17.3. Disabling TLS for a container registry to use as insecure registry Copiar enlaceEnlace copiado en el portapapeles!
You can disable TLS (transport layer security) for one or more container registries by editing the
insecureRegistries
HyperConverged
Prerequisites
-
Log in to the cluster as a user with the role.
cluster-admin
Procedure
Edit the
custom resource and add a list of insecure registries to theHyperConvergedfield.spec.storageImport.insecureRegistriesapiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: storageImport: insecureRegistries:1 - "private-registry-example-1:5000" - "private-registry-example-2:5000"- 1
- Replace the examples in this list with valid registry hostnames.
8.18.17.4. Next steps Copiar enlaceEnlace copiado en el portapapeles!
- Import the container disk into persistent storage for a virtual machine.
-
Create a virtual machine that uses a volume for ephemeral storage.
containerDisk
8.18.18. Preparing CDI scratch space Copiar enlaceEnlace copiado en el portapapeles!
8.18.18.1. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.18.18.2. Understanding scratch space Copiar enlaceEnlace copiado en el portapapeles!
The Containerized Data Importer (CDI) requires scratch space (temporary storage) to complete some operations, such as importing and uploading virtual machine images. During this process, CDI provisions a scratch space PVC equal to the size of the PVC backing the destination data volume (DV). The scratch space PVC is deleted after the operation completes or aborts.
You can define the storage class that is used to bind the scratch space PVC in the
spec.scratchSpaceStorageClass
HyperConverged
If the defined storage class does not match a storage class in the cluster, then the default storage class defined for the cluster is used. If there is no default storage class defined in the cluster, the storage class used to provision the original DV or PVC is used.
CDI requires requesting scratch space with a
file
block
file
Manual provisioning
If there are no storage classes, CDI uses any PVCs in the project that match the size requirements for the image. If there are no PVCs that match these requirements, the CDI import pod remains in a Pending state until an appropriate PVC is made available or until a timeout function kills the pod.
8.18.18.3. CDI operations that require scratch space Copiar enlaceEnlace copiado en el portapapeles!
| Type | Reason |
|---|---|
| Registry imports | CDI must download the image to a scratch space and extract the layers to find the image file. The image file is then passed to QEMU-IMG for conversion to a raw disk. |
| Upload image | QEMU-IMG does not accept input from STDIN. Instead, the image to upload is saved in scratch space before it can be passed to QEMU-IMG for conversion. |
| HTTP imports of archived images | QEMU-IMG does not know how to handle the archive formats CDI supports. Instead, the image is unarchived and saved into scratch space before it is passed to QEMU-IMG. |
| HTTP imports of authenticated images | QEMU-IMG inadequately handles authentication. Instead, the image is saved to scratch space and authenticated before it is passed to QEMU-IMG. |
| HTTP imports of custom certificates | QEMU-IMG inadequately handles custom certificates of HTTPS endpoints. Instead, CDI downloads the image to scratch space before passing the file to QEMU-IMG. |
8.18.18.4. Defining a storage class Copiar enlaceEnlace copiado en el portapapeles!
You can define the storage class that the Containerized Data Importer (CDI) uses when allocating scratch space by adding the
spec.scratchSpaceStorageClass
HyperConverged
Prerequisites
-
Install the OpenShift CLI ().
oc
Procedure
Edit the
CR by running the following command:HyperConverged$ oc edit hco -n openshift-cnv kubevirt-hyperconvergedAdd the
field to the CR, setting the value to the name of a storage class that exists in the cluster:spec.scratchSpaceStorageClassapiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged spec: scratchSpaceStorageClass: "<storage_class>"1 - 1
- If you do not specify a storage class, CDI uses the storage class of the persistent volume claim that is being populated.
-
Save and exit your default editor to update the CR.
HyperConverged
8.18.18.5. CDI supported operations matrix Copiar enlaceEnlace copiado en el portapapeles!
This matrix shows the supported CDI operations for content types against endpoints, and which of these operations requires scratch space.
| Content types | HTTP | HTTPS | HTTP basic auth | Registry | Upload |
|---|---|---|---|---|---|
| KubeVirt (QCOW2) |
✓ QCOW2 |
✓ QCOW2** |
✓ QCOW2 |
✓ QCOW2* |
✓ QCOW2* |
| KubeVirt (RAW) |
✓ RAW |
✓ RAW |
✓ RAW |
✓ RAW* |
✓ RAW* |
✓ Supported operation
□ Unsupported operation
* Requires scratch space
** Requires scratch space if a custom certificate authority is required
8.18.19. Re-using persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
To re-use a statically provisioned persistent volume (PV), you must first reclaim the volume. This involves deleting the PV so that the storage configuration can be re-used.
8.18.19.1. About reclaiming statically provisioned persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
When you reclaim a persistent volume (PV), you unbind the PV from a persistent volume claim (PVC) and delete the PV. Depending on the underlying storage, you might need to manually delete the shared storage.
You can then re-use the PV configuration to create a PV with a different name.
Statically provisioned PVs must have a reclaim policy of
Retain
The
Recycle
8.18.19.2. Reclaiming statically provisioned persistent volumes Copiar enlaceEnlace copiado en el portapapeles!
Reclaim a statically provisioned persistent volume (PV) by unbinding the persistent volume claim (PVC) and deleting the PV. You might also need to manually delete the shared storage.
Reclaiming a statically provisioned PV is dependent on the underlying storage. This procedure provides a general approach that might need to be customized depending on your storage.
Procedure
Ensure that the reclaim policy of the PV is set to
:RetainCheck the reclaim policy of the PV:
$ oc get pv <pv_name> -o yaml | grep 'persistentVolumeReclaimPolicy'If the
is not set topersistentVolumeReclaimPolicy, edit the reclaim policy with the following command:Retain$ oc patch pv <pv_name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
Ensure that no resources are using the PV:
$ oc describe pvc <pvc_name> | grep 'Mounted By:'Remove any resources that use the PVC before continuing.
Delete the PVC to release the PV:
$ oc delete pvc <pvc_name>Optional: Export the PV configuration to a YAML file. If you manually remove the shared storage later in this procedure, you can refer to this configuration. You can also use
parameters in this file as the basis to create a new PV with the same storage configuration after you reclaim the PV:spec$ oc get pv <pv_name> -o yaml > <file_name>.yamlDelete the PV:
$ oc delete pv <pv_name>Optional: Depending on the storage type, you might need to remove the contents of the shared storage folder:
$ rm -rf <path_to_share_storage>Optional: Create a PV that uses the same storage configuration as the deleted PV. If you exported the reclaimed PV configuration earlier, you can use the
parameters of that file as the basis for a new PV manifest:specNoteTo avoid possible conflict, it is good practice to give the new PV object a different name than the one that you deleted.
$ oc create -f <new_pv_name>.yaml
8.18.20. Deleting data volumes Copiar enlaceEnlace copiado en el portapapeles!
You can manually delete a data volume by using the
oc
When you delete a virtual machine, the data volume it uses is automatically deleted.
8.18.20.1. About data volumes Copiar enlaceEnlace copiado en el portapapeles!
DataVolume
8.18.20.2. Listing all data volumes Copiar enlaceEnlace copiado en el portapapeles!
You can list the data volumes in your cluster by using the
oc
Procedure
List all data volumes by running the following command:
$ oc get dvs
8.18.20.3. Deleting a data volume Copiar enlaceEnlace copiado en el portapapeles!
You can delete a data volume by using the
oc
Prerequisites
- Identify the name of the data volume that you want to delete.
Procedure
Delete the data volume by running the following command:
$ oc delete dv <datavolume_name>NoteThis command only deletes objects that exist in the current project. Specify the
option if the object you want to delete is in a different project or namespace.-n <project_name>
Chapter 9. Virtual machine templates Copiar enlaceEnlace copiado en el portapapeles!
9.1. Creating virtual machine templates Copiar enlaceEnlace copiado en el portapapeles!
9.1.1. About virtual machine templates Copiar enlaceEnlace copiado en el portapapeles!
Preconfigured Red Hat virtual machine templates are listed in the Templates tab within the Virtualization page. These templates are available for different versions of Red Hat Enterprise Linux, Fedora, Microsoft Windows 10, and Microsoft Windows Servers. Each Red Hat virtual machine template is preconfigured with the operating system image, default settings for the operating system, flavor (CPU and memory), and workload type (server).
The Templates tab displays four types of virtual machine templates:
- Red Hat Supported templates are fully supported by Red Hat.
- User Supported templates are Red Hat Supported templates that were cloned and created by users.
- Red Hat Provided templates have limited support from Red Hat.
- User Provided templates are Red Hat Provided templates that were cloned and created by users.
In the Templates tab, you cannot edit or delete Red Hat Supported or Red Hat Provided templates. You can only edit or delete custom virtual machine templates that were created by users.
Using a Red Hat template is convenient because the template is already preconfigured. When you select a Red Hat template to create your own custom template, the Create Virtual Machine Template wizard prompts you to add a boot source if a boot source was not added previously. Then, you can either save your custom template or continue to customize it and save it.
You can also select the Create Virtual Machine Template wizard directly and create a custom virtual machine template. The wizard prompts you to provide configuration details for the operating system, flavor, workload type, and other settings. You can add a boot source and continue to customize your template and save it.
9.1.2. About virtual machines and boot sources Copiar enlaceEnlace copiado en el portapapeles!
Virtual machines consist of a virtual machine definition and one or more disks that are backed by data volumes. Virtual machine templates enable you to create virtual machines using predefined virtual machine specifications.
Every virtual machine template requires a boot source, which is a fully configured virtual machine disk image including configured drivers. Each virtual machine template contains a virtual machine definition with a pointer to the boot source. Each boot source has a predefined name and namespace. For some operating systems, a boot source is automatically provided. If it is not provided, then an administrator must prepare a custom boot source.
Provided boot sources are updated automatically to the latest version of the operating system. For auto-updated boot sources, persistent volume claims (PVCs) are created with the cluster’s default storage class. If you select a different default storage class after configuration, you must delete the existing data volumes in the cluster namespace that are configured with the previous default storage class.
To use the boot sources feature, install the latest release of OpenShift Virtualization. The namespace
openshift-virtualization-os-images
Define a boot source by using a persistent volume claim (PVC) that is populated by uploading a local file, cloning an existing PVC, importing from a registry, or by URL. Attach a boot source to a virtual machine template by using the web console. After the boot source is attached to a virtual machine template, you create any number of fully configured ready-to-use virtual machines from the template.
9.1.3. Adding a boot source for a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
A boot source can be configured for any virtual machine template that you want to use for creating virtual machines or custom templates. When virtual machine templates are configured with a boot source, they are labeled Available in the Templates tab. After you add a boot source to a template, you can create a new virtual machine from the template.
There are four methods for selecting and adding a boot source in the web console:
- Upload local file (creates PVC)
- Import via URL (creates PVC)
- Clone existing PVC (creates PVC)
- Import via Registry (creates PVC)
Prerequisites
-
To add a boot source, you must be logged in as a user with the RBAC role or as an administrator. You do not need special privileges to create a virtual machine from a template with a boot source added.
os-images.kubevirt.io:edit - To upload a local file, the operating system image file must exist on your local machine.
- To import via URL, access to the web server with the operating system image is required. For example: the Red Hat Enterprise Linux web page with images.
- To clone an existing PVC, access to the project with a PVC is required.
- To import via registry, access to the container registry is required.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Identify the virtual machine template for which you want to configure a boot source and click Add source.
- In the Add boot source to template window, click Select boot source, select a method for creating a persistent volume claim (PVC): Upload local file, Import via URL, Clone existing PVC, or Import via Registry.
- Optional: Click This is a CD-ROM boot source to mount a CD-ROM and use it to install the operating system on to an empty disk. The additional empty disk is automatically created and mounted by OpenShift Virtualization. If the additional disk is not needed, you can remove it when you create the virtual machine.
Enter a value for Persistent Volume Claim size to specify the PVC size that is adequate for the uncompressed image and any additional space that is required.
- Optional: Enter a name for Source provider to associate the name with this template.
Optional: Advanced Storage settings: Click Storage class and select the storage class that is used to create the disk. Typically, this storage class is the default storage class that is created for use by all PVCs.
NoteProvided boot sources are updated automatically to the latest version of the operating system. For auto-updated boot sources, persistent volume claims (PVCs) are created with the cluster’s default storage class. If you select a different default storage class after configuration, you must delete the existing data volumes in the cluster namespace that are configured with the previous default storage class.
Optional: Advanced Storage settings: Click Access mode and select an access mode for the persistent volume:
- Single User (RWO) mounts the volume as read-write by a single node.
- Shared Access (RWX) mounts the volume as read-write by many nodes.
- Read Only (ROX) mounts the volume as read-only by many nodes.
- Optional: Advanced Storage settings: Click Volume mode if you want to select Block instead of the default value Filesystem. OpenShift Virtualization can statically provision raw block volumes. These volumes do not have a file system, and can provide performance benefits for applications that either write to the disk directly or implement their own storage service.
Select the appropriate method to save your boot source:
- Click Save and upload if you uploaded a local file.
- Click Save and import if you imported content from a URL or the registry.
- Click Save and clone if you cloned an existing PVC.
Your custom virtual machine template with a boot source is listed in the Templates tab, and you can create virtual machines by using this template.
9.1.3.1. Virtual machine template fields for adding a boot source Copiar enlaceEnlace copiado en el portapapeles!
The following table describes the fields for Add boot source to template window. This window displays when you click Add Source for a virtual machine template in the Templates tab.
| Name | Parameter | Description |
|---|---|---|
| Boot source type | Upload local file (creates PVC) | Upload a file from your local device. Supported file types include gz, xz, tar, and qcow2. |
| Import via URL (creates PVC) | Import content from an image available from an HTTP or HTTPS endpoint. Obtain the download link URL from the web page where the image download is available and enter that URL link in the Import via URL (creates PVC) field. Example: For a Red Hat Enterprise Linux image, log on to the Red Hat Customer Portal, access the image download page, and copy the download link URL for the KVM guest image. | |
| Clone existing PVC (creates PVC) | Use a PVC that is already available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Specify the bootable operating system container that is located in a registry and accessible from the cluster. Example: kubevirt/cirros-registry-dis-demo. | |
| Source provider | Optional field. Add descriptive text about the source for the template or the name of the user who created the template. Example: Red Hat. | |
| Advanced | Storage class | The storage class that is used to create the disk. |
| Access mode | Access mode of the persistent volume. Supported access modes are Single User (RWO), Shared Access (RWX), Read Only (ROX). If Single User (RWO) is selected, the disk can be mounted as read/write by a single node. If Shared Access (RWX) is selected, the disk can be mounted as read-write by many nodes. The
+ Note Shared Access (RWX) is required for some features, such as live migration of virtual machines between nodes. | |
| Volume mode | Defines whether the persistent volume uses a formatted file system or raw block state. Supported modes are Block and Filesystem. The
|
9.1.4. Marking virtual machine templates as favorites Copiar enlaceEnlace copiado en el portapapeles!
For easier access to virtual machine templates that are used frequently, you can mark those templates as favorites.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Identify the Red Hat template that you want to mark as a favorite.
-
Click the Options menu
and select Favorite template. The template moves up higher in the list of displayed templates.
9.1.5. Filtering the list of virtual machine templates by providers Copiar enlaceEnlace copiado en el portapapeles!
In the Templates tab, you can use the Search by name field to search for virtual machine templates by specifying either the name of the template or a label that identfies the template. You can also filter templates by the provider, and display only those templates that meet your filtering criteria.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- To filter templates, click Filter.
- Select the appropriate checkbox from the list to filter the templates: Red Hat Supported, User Supported, Red Hat Provided, and User Provided.
9.1.6. Creating a virtual machine template with the wizard in the web console Copiar enlaceEnlace copiado en el portapapeles!
The web console features the Create Virtual Machine Template wizard that guides you through the General, Networking, Storage, Advanced, and Review steps to simplify the process of creating virtual machine templates. All required fields are marked with a *. The Create Virtual Machine Template wizard prevents you from moving to the next step until you provide values in the required fields.
The wizard guides you to create a custom virtual machine template where you specify the operating system, boot source, flavor, and other settings.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Click Create and select Template with Wizard.
- Fill in all required fields in the General step.
Click Next to progress to the Networking step. A NIC that is named
is attached by default.nic0- Optional: Click Add Network Interface to create additional NICs.
-
Optional: You can remove any or all NICs by clicking the Options menu
and selecting Delete. Virtual machines created from a template do not need a NIC attached. NICs can be created after a virtual machine has been created.
- Click Next to progress to the Storage step.
Click Add Disk to add a disk, and complete your selections for the fields in the Add Disk screen.
NoteIf Import via URL (creates PVC), Import via Registry (creates PVC), or Container (ephemeral) is selected as Source, a
disk is created and attached to the virtual machine as the Bootable Disk.rootdiskA Bootable Disk is not required for virtual machines provisioned from a PXE source if there are no disks attached to the virtual machine. If one or more disks are attached to the virtual machine, you must select one as the Bootable Disk.
Blank disks, PVC disks without a valid boot source, and the cloudinitdisk cannot be used as a boot source.
Optional: Click Advanced to configure cloud-init and SSH access.
NoteStatically inject an SSH key by using the custom script in cloud-init or in the wizard. This allows you to securely and remotely manage virtual machines and manage and transfer information. This step is strongly recommended to secure your VM.
- Click Review to review and confirm your settings.
- Click Create Virtual Machine template.
- Click See virtual machine template details to view details about the virtual machine template.
The template is also listed in the Templates tab.
9.1.7. Virtual machine template wizard fields Copiar enlaceEnlace copiado en el portapapeles!
The following tables describe the fields for the General, Networking, Storage, and Advanced steps in the Create Virtual Machine Template wizard.
9.1.7.1. Virtual machine template wizard fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Parameter | Description |
|---|---|---|
| Template | Template from which to create the virtual machine. Selecting a template will automatically complete other fields. | |
| Name | The name can contain lowercase letters (
| |
| Template provider | The name of the user who is creating the template for the cluster or any meaningful name that identifies this template. | |
| Template support | No additional support | This template does not have additional support in the cluster. |
| Support by template provider | This template is supported by the template provider. | |
| Description | Optional description field. | |
| Operating System | The operating system that is selected for the virtual machine. Selecting an operating system automatically selects the default Flavor and Workload Type for that operating system. | |
| Boot Source | Import via URL (creates PVC) | Import content from an image available from an HTTP or HTTPS endpoint. Example: Obtaining a URL link from the web page with the operating system image. |
| Clone existing PVC (creates PVC) | Select an existent persistent volume claim available on the cluster and clone it. | |
| Import via Registry (creates PVC) | Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Example:
| |
| PXE (network boot - adds network interface) | Boot an operating system from a server on the network. Requires a PXE bootable network attachment definition. | |
| Persistent Volume Claim project | Project name that you want to use for cloning the PVC. | |
| Persistent Volume Claim name | PVC name that should apply to this virtual machine template if you are cloning an existing PVC. | |
| Mount this as a CD-ROM boot source | A CD-ROM requires an additional disk for installing the operating system. Select the checkbox to add a disk and customize it later. | |
| Flavor | Tiny, Small, Medium, Large, Custom | Presets the amount of CPU and memory in a virtual machine template with predefined values that are allocated to the virtual machine, depending on the operating system associated with that template. If you choose a default template, you can override the
|
| Workload Type Note If you choose the incorrect Workload Type, there could be performance or resource utilization issues (such as a slow UI). | Desktop | A virtual machine configuration for use on a desktop. Ideal for consumption on a small scale. Recommended for use with the web console. |
| Server | Balances performance and it is compatible with a wide range of server workloads. | |
| High-Performance | A virtual machine configuration that is optimized for high-performance workloads. |
9.1.7.2. Networking fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Name | Name for the network interface controller. |
| Model | Indicates the model of the network interface controller. Supported values are e1000e and virtio. |
| Network | List of available network attachment definitions. |
| Type | List of available binding methods. For the default pod network,
|
| MAC Address | MAC address for the network interface controller. If a MAC address is not specified, one is assigned automatically. |
9.1.7.3. Storage fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Selection | Description |
|---|---|---|
| Source | Blank (creates PVC) | Create an empty disk. |
| Import via URL (creates PVC) | Import content via URL (HTTP or HTTPS endpoint). | |
| Use an existing PVC | Use a PVC that is already available in the cluster. | |
| Clone existing PVC (creates PVC) | Select an existing PVC available in the cluster and clone it. | |
| Import via Registry (creates PVC) | Import content via container registry. | |
| Container (ephemeral) | Upload content from a container located in a registry accessible from the cluster. The container disk should be used only for read-only filesystems such as CD-ROMs or temporary virtual machines. | |
| Name | Name of the disk. The name can contain lowercase letters (
| |
| Size | Size of the disk in GiB. | |
| Type | Type of disk. Example: Disk or CD-ROM | |
| Interface | Type of disk device. Supported interfaces are virtIO, SATA, and SCSI. | |
| Storage Class | The storage class that is used to create the disk. | |
| Advanced → Volume Mode | Defines whether the persistent volume uses a formatted file system or raw block state. Default is Filesystem. | |
| Advanced → Access Mode | Access mode of the persistent volume. Supported access modes are Single User (RWO), Shared Access (RWX), and Read Only (ROX). |
Advanced storage settings
The following advanced storage settings are available for Blank, Import via URL, and Clone existing PVC disks. These parameters are optional. If you do not specify these parameters, the system uses the default values from the
kubevirt-storage-class-defaults
| Name | Parameter | Description |
|---|---|---|
| Volume Mode | Filesystem | Stores the virtual disk on a file system-based volume. |
| Block | Stores the virtual disk directly on the block volume. Only use
| |
| Access Mode | Single User (RWO) | The disk can be mounted as read/write by a single node. |
| Shared Access (RWX) | The disk can be mounted as read/write by many nodes. Note This is required for some features, such as live migration of virtual machines between nodes. | |
| Read Only (ROX) | The disk can be mounted as read-only by many nodes. |
9.1.7.4. Cloud-init fields Copiar enlaceEnlace copiado en el portapapeles!
| Name | Description |
|---|---|
| Hostname | Sets a specific hostname for the virtual machine. |
| Authorized SSH Keys | The user’s public key that is copied to ~/.ssh/authorized_keys on the virtual machine. |
| Custom script | Replaces other options with a field in which you paste a custom cloud-init script. |
9.2. Editing virtual machine templates Copiar enlaceEnlace copiado en el portapapeles!
You can update a virtual machine template in the web console, either by editing the full configuration in the YAML editor or by selecting a custom template in the Templates tab and modifying the editable items.
9.2.1. Editing a virtual machine template in the web console Copiar enlaceEnlace copiado en el portapapeles!
Edit select values of a virtual machine template in the web console by clicking the pencil icon next to the relevant field. Other values can be edited using the CLI.
Labels and annotations are editable for both preconfigured Red Hat templates and your custom virtual machine templates. All other values are editable only for custom virtual machine templates that users have created using the Red Hat templates or the Create Virtual Machine Template wizard.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Select a virtual machine template.
- Click the VM Template Details tab.
- Click the pencil icon to make a field editable.
- Make the relevant changes and click Save.
Editing a virtual machine template will not affect virtual machines already created from that template.
9.2.2. Editing virtual machine template YAML configuration in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can edit the YAML configuration of a virtual machine template from the web console.
Some parameters cannot be modified. If you click Save with an invalid configuration, an error message indicates the parameter that cannot be modified.
Navigating away from the YAML screen while editing cancels any changes to the configuration that you made.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Select a template to open the VM Template Details screen.
- Click the YAML tab to display the editable configuration.
- Edit the file and click Save.
A confirmation message, which includes the updated version number for the object, shows that the YAML configuration was successfully edited.
9.2.3. Adding a virtual disk to a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
Use this procedure to add a virtual disk to a virtual machine template.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Select a virtual machine template to open the VM Template Details screen.
- Click the Disks tab.
In the Add Disk window, specify the Source, Name, Size, Type, Interface, and Storage Class.
-
Optional: In the Advanced list, specify the Volume Mode and Access Mode for the virtual disk. If you do not specify these parameters, the system uses the default values from the config map.
kubevirt-storage-class-defaults
-
Optional: In the Advanced list, specify the Volume Mode and Access Mode for the virtual disk. If you do not specify these parameters, the system uses the default values from the
- Click Add.
9.2.4. Adding a network interface to a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
Use this procedure to add a network interface to a virtual machine template.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Select a virtual machine template to open the VM Template Details screen.
- Click the Network Interfaces tab.
- Click Add Network Interface.
- In the Add Network Interface window, specify the Name, Model, Network, Type, and MAC Address of the network interface.
- Click Add.
9.2.5. Editing CD-ROMs for Templates Copiar enlaceEnlace copiado en el portapapeles!
Use the following procedure to edit CD-ROMs for virtual machine templates.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Templates tab.
- Select a virtual machine template to open the VM Template Details screen.
- Click the Disks tab.
-
Click the Options menu
for the CD-ROM that you want to edit and select Edit.
- In the Edit CD-ROM window, edit the fields: Source, Persistent Volume Claim, Name, Type, and Interface.
- Click Save.
9.3. Enabling dedicated resources for virtual machine templates Copiar enlaceEnlace copiado en el portapapeles!
Virtual machines can have resources of a node, such as CPU, dedicated to them to improve performance.
9.3.1. About dedicated resources Copiar enlaceEnlace copiado en el portapapeles!
When you enable dedicated resources for your virtual machine, your virtual machine’s workload is scheduled on CPUs that will not be used by other processes. By using dedicated resources, you can improve the performance of the virtual machine and the accuracy of latency predictions.
9.3.2. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
The CPU Manager must be configured on the node. Verify that the node has the =
cpumanagerlabel before scheduling virtual machine workloads.true
9.3.3. Enabling dedicated resources for a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
You can enable dedicated resources for a virtual machine template in the Details tab. Virtual machines that were created by using a Red Hat template or the wizard can be enabled with dedicated resources.
Procedure
- Click Workloads → Virtual Machine Templates from the side menu.
- Select a virtual machine template to open the Virtual Machine Template tab.
- Click the Details tab.
- Click the pencil icon to the right of the Dedicated Resources field to open the Dedicated Resources window.
- Select Schedule this workload with dedicated resources (guaranteed policy).
- Click Save.
9.4. Deleting a virtual machine template Copiar enlaceEnlace copiado en el portapapeles!
Red Hat virtual machine templates cannot be deleted. You can use the web console to delete:
- Virtual machine templates created from Red Hat templates
- Custom virtual machine templates that were created by using the Create Virtual Machine Template wizard.
9.4.1. Deleting a virtual machine template in the web console Copiar enlaceEnlace copiado en el portapapeles!
Deleting a virtual machine template permanently removes it from the cluster.
You can delete virtual machine templates that were created by using a Red Hat template or the Create Virtual Machine Template wizard. Preconfigured virtual machine templates that are provided by Red Hat cannot be deleted.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
Click the Templates tab. Select the appropriate method to delete a virtual machine template:
-
Click the Options menu
of the template to delete and select Delete Template.
- Click the template name to open the Virtual Machine Template Details screen and click Actions → Delete Template.
-
Click the Options menu
- In the confirmation pop-up window, click Delete to permanently delete the template.
Chapter 10. Live migration Copiar enlaceEnlace copiado en el portapapeles!
10.1. Virtual machine live migration Copiar enlaceEnlace copiado en el portapapeles!
10.1.1. Understanding live migration Copiar enlaceEnlace copiado en el portapapeles!
Live migration is the process of moving a running virtual machine instance (VMI) to another node in the cluster without interrupting the virtual workload or access. If a VMI uses the
LiveMigrate
Virtual machines must have a persistent volume claim (PVC) with a shared ReadWriteMany (RWX) access mode to be live migrated.
Live migration is supported for virtual machines that are attached to an SR-IOV network interface only if the
sriovLiveMigration
spec.featureGates.sriovLiveMigration
true
virt-launcher
SYS_RESOURCE
10.1.2. Updating access mode for live migration Copiar enlaceEnlace copiado en el portapapeles!
For live migration to function properly, you must use the ReadWriteMany (RWX) access mode. Use this procedure to update the access mode, if needed.
Procedure
To set the RWX access mode, run the following
command:oc patch$ oc patch -n openshift-cnv \ cm kubevirt-storage-class-defaults \ -p '{"data":{"'$<STORAGE_CLASS>'.accessMode":"ReadWriteMany"}}'
10.2. Live migration limits and timeouts Copiar enlaceEnlace copiado en el portapapeles!
Apply live migration limits and timeouts so that migration processes do not overwhelm the cluster. Configure these settings by editing the
HyperConverged
10.2.1. Configuring live migration limits and timeouts Copiar enlaceEnlace copiado en el portapapeles!
Configure live migration limits and timeouts for the cluster by updating the
HyperConverged
openshift-cnv
Procedure
Edit the
CR and add the necessary live migration parameters.HyperConverged$ oc edit hco -n openshift-cnv kubevirt-hyperconvergedExample configuration file
apiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: liveMigrationConfig:1 bandwidthPerMigration: 64Mi completionTimeoutPerGiB: 800 parallelMigrationsPerCluster: 5 parallelOutboundMigrationsPerNode: 2 progressTimeout: 150- 1
- In this example, the
spec.liveMigrationConfigarray contains the default values for each field.
NoteYou can restore the default value for any
field by deleting that key/value pair and saving the file. For example, deletespec.liveMigrationConfigto restore the defaultprogressTimeout: <value>.progressTimeout: 150
10.2.2. Cluster-wide live migration limits and timeouts Copiar enlaceEnlace copiado en el portapapeles!
| Parameter | Description | Default |
|---|---|---|
|
| Number of migrations running in parallel in the cluster. | 5 |
|
| Maximum number of outbound migrations per node. | 2 |
|
| Bandwidth limit of each migration, in MiB/s. | 0 [1] |
|
| The migration is canceled if it has not completed in this time, in seconds per GiB of memory. For example, a virtual machine instance with 6GiB memory times out if it has not completed migration in 4800 seconds. If the
| 800 |
|
| The migration is canceled if memory copy fails to make progress in this time, in seconds. | 150 |
-
The default value of is unlimited.
0
10.3. Migrating a virtual machine instance to another node Copiar enlaceEnlace copiado en el portapapeles!
Manually initiate a live migration of a virtual machine instance to another node using either the web console or the CLI.
10.3.1. Initiating live migration of a virtual machine instance in the web console Copiar enlaceEnlace copiado en el portapapeles!
Migrate a running virtual machine instance to a different node in the cluster.
The Migrate Virtual Machine action is visible to all users but only admin users can initiate a virtual machine migration.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
You can initiate the migration from this screen, which makes it easier to perform actions on multiple virtual machines in the one screen, or from the Virtual Machine Overview screen where you can view comprehensive details of the selected virtual machine:
-
Click the Options menu
at the end of virtual machine and select Migrate Virtual Machine.
- Click the virtual machine name to open the Virtual Machine Overview screen and click Actions → Migrate Virtual Machine.
-
Click the Options menu
- Click Migrate to migrate the virtual machine to another node.
10.3.2. Initiating live migration of a virtual machine instance in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Initiate a live migration of a running virtual machine instance by creating a
VirtualMachineInstanceMigration
Procedure
Create a
configuration file for the virtual machine instance to migrate. For example,VirtualMachineInstanceMigration:vmi-migrate.yamlapiVersion: kubevirt.io/v1 kind: VirtualMachineInstanceMigration metadata: name: migration-job spec: vmiName: vmi-fedoraCreate the object in the cluster by running the following command:
$ oc create -f vmi-migrate.yaml
The
VirtualMachineInstanceMigration
10.4. Monitoring live migration of a virtual machine instance Copiar enlaceEnlace copiado en el portapapeles!
You can monitor the progress of a live migration of a virtual machine instance from either the web console or the CLI.
10.4.1. Monitoring live migration of a virtual machine instance in the web console Copiar enlaceEnlace copiado en el portapapeles!
For the duration of the migration, the virtual machine has a status of Migrating. This status is displayed in the Virtual Machines tab or in the Virtual Machine Overview screen for the migrating virtual machine.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
10.4.2. Monitoring live migration of a virtual machine instance in the CLI Copiar enlaceEnlace copiado en el portapapeles!
The status of the virtual machine migration is stored in the
Status
VirtualMachineInstance
Procedure
Use the
command on the migrating virtual machine instance:oc describe$ oc describe vmi vmi-fedoraExample output
... Status: Conditions: Last Probe Time: <nil> Last Transition Time: <nil> Status: True Type: LiveMigratable Migration Method: LiveMigration Migration State: Completed: true End Timestamp: 2018-12-24T06:19:42Z Migration UID: d78c8962-0743-11e9-a540-fa163e0c69f1 Source Node: node2.example.com Start Timestamp: 2018-12-24T06:19:35Z Target Node: node1.example.com Target Node Address: 10.9.0.18:43891 Target Node Domain Detected: true
10.5. Cancelling the live migration of a virtual machine instance Copiar enlaceEnlace copiado en el portapapeles!
Cancel the live migration so that the virtual machine instance remains on the original node.
You can cancel a live migration from either the web console or the CLI.
10.5.1. Cancelling live migration of a virtual machine instance in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can cancel a live migration of the virtual machine instance using the Options menu
found on each virtual machine in the Virtualization → Virtual Machines tab, or from the Actions menu available on all tabs in the Virtual Machine Overview screen.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
You can cancel the migration from this screen, which makes it easier to perform actions on multiple virtual machines, or from the Virtual Machine Overview screen where you can view comprehensive details of the selected virtual machine:
-
Click the Options menu
at the end of virtual machine and select Cancel Virtual Machine Migration.
- Select a virtual machine name to open the Virtual Machine Overview screen and click Actions → Cancel Virtual Machine Migration.
-
Click the Options menu
- Click Cancel Migration to cancel the virtual machine live migration.
10.5.2. Cancelling live migration of a virtual machine instance in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Cancel the live migration of a virtual machine instance by deleting the
VirtualMachineInstanceMigration
Procedure
Delete the
object that triggered the live migration,VirtualMachineInstanceMigrationin this example:migration-job$ oc delete vmim migration-job
10.6. Configuring virtual machine eviction strategy Copiar enlaceEnlace copiado en el portapapeles!
The
LiveMigrate
10.6.1. Configuring custom virtual machines with the LiveMigration eviction strategy Copiar enlaceEnlace copiado en el portapapeles!
You only need to configure the
LiveMigration
Procedure
Add the
option to theevictionStrategy: LiveMigratesection in the virtual machine configuration file. This example usesspec.template.specto update the relevant snippet of theoc editconfiguration file:VirtualMachine$ oc edit vm <custom-vm> -n <my-namespace>apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: custom-vm spec: template: spec: evictionStrategy: LiveMigrate ...Restart the virtual machine for the update to take effect:
$ virtctl restart <custom-vm> -n <my-namespace>
Chapter 11. Node maintenance Copiar enlaceEnlace copiado en el portapapeles!
11.1. About node maintenance Copiar enlaceEnlace copiado en el portapapeles!
11.1.1. Understanding node maintenance mode Copiar enlaceEnlace copiado en el portapapeles!
Nodes can be placed into maintenance mode using the
oc adm
NodeMaintenance
Placing a node into maintenance marks the node as unschedulable and drains all the virtual machines and pods from it. Virtual machine instances that have a
LiveMigrate
Virtual machine instances without an eviction strategy are shut down. Virtual machines with a
RunStrategy
Running
RerunOnFailure
RunStrategy
Manual
Virtual machines must have a persistent volume claim (PVC) with a shared
ReadWriteMany
When installed as part of OpenShift Virtualization, Node Maintenance Operator watches for new or deleted
NodeMaintenance
NodeMaintenance
NodeMaintenance
Using a
NodeMaintenance
oc adm cordon
oc adm drain
11.1.2. Maintaining bare metal nodes Copiar enlaceEnlace copiado en el portapapeles!
When you deploy OpenShift Container Platform on bare metal infrastructure, there are additional considerations that must be taken into account compared to deploying on cloud infrastructure. Unlike in cloud environments where the cluster nodes are considered ephemeral, re-provisioning a bare metal node requires significantly more time and effort for maintenance tasks.
When a bare metal node fails, for example, if a fatal kernel error happens or a NIC card hardware failure occurs, workloads on the failed node need to be restarted elsewhere else on the cluster while the problem node is repaired or replaced. Node maintenance mode allows cluster administrators to gracefully power down nodes, moving workloads to other parts of the cluster and ensuring workloads do not get interrupted. Detailed progress and node status details are provided during maintenance.
11.2. Setting a node to maintenance mode Copiar enlaceEnlace copiado en el portapapeles!
Place a node into maintenance from the web console, CLI, or using a
NodeMaintenance
11.2.1. Setting a node to maintenance mode in the web console Copiar enlaceEnlace copiado en el portapapeles!
Set a node to maintenance mode using the Options menu
found on each node in the Compute → Nodes list, or using the Actions control of the Node Details screen.
Procedure
- In the OpenShift Virtualization console, click Compute → Nodes.
You can set the node to maintenance from this screen, which makes it easier to perform actions on multiple nodes in the one screen or from the Node Details screen where you can view comprehensive details of the selected node:
-
Click the Options menu
at the end of the node and select Start Maintenance.
- Click the node name to open the Node Details screen and click Actions → Start Maintenance.
-
Click the Options menu
- Click Start Maintenance in the confirmation window.
The node will live migrate virtual machine instances that have the
LiveMigration
11.2.2. Setting a node to maintenance mode in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Set a node to maintenance mode by marking it as unschedulable and using the
oc adm drain
Procedure
Mark the node as unschedulable. The node status changes to
.NotReady,SchedulingDisabled$ oc adm cordon <node1>Drain the node in preparation for maintenance. The node live migrates virtual machine instances that have the
condition set toLiveMigratableand theTruefield set tospec:evictionStrategy. All other pods and virtual machines on the node are deleted and recreated on another node.LiveMigrate$ oc adm drain <node1> --delete-emptydir-data --ignore-daemonsets=true --force-
The flag removes any virtual machine instances on the node that use
--delete-emptydir-datavolumes. Data in these volumes is ephemeral and is safe to be deleted after termination.emptyDir -
The flag ensures that daemon sets are ignored and pod eviction can continue successfully.
--ignore-daemonsets=true -
The flag is required to delete pods that are not managed by a replica set or daemon set controller.
--force
-
The
11.2.3. Setting a node to maintenance mode with a NodeMaintenance custom resource Copiar enlaceEnlace copiado en el portapapeles!
You can put a node into maintenance mode with a
NodeMaintenance
NodeMaintenance
Prerequisites
-
Install the OpenShift Container Platform CLI .
oc -
Log in to the cluster as a user with privileges.
cluster-admin
Procedure
Create the following node maintenance CR, and save the file as
:nodemaintenance-cr.yamlapiVersion: nodemaintenance.kubevirt.io/v1beta1 kind: NodeMaintenance metadata: name: maintenance-example1 spec: nodeName: node-1.example.com2 reason: "Node maintenance"3 Apply the node maintenance schedule by running the following command:
$ oc apply -f nodemaintenance-cr.yamlCheck the progress of the maintenance task by running the following command, replacing
with the name of your node:<node-name>$ oc describe node <node-name>Example output
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal NodeNotSchedulable 61m kubelet Node node-1.example.com status is now: NodeNotSchedulable
11.2.3.1. Checking status of current NodeMaintenance CR tasks Copiar enlaceEnlace copiado en el portapapeles!
You can check the status of current
NodeMaintenance
Prerequisites
-
Install the OpenShift Container Platform CLI .
oc -
Log in as a user with privileges.
cluster-admin
Procedure
Check the status of current node maintenance tasks by running the following command:
$ oc get NodeMaintenance -o yamlExample output
apiVersion: v1 items: - apiVersion: nodemaintenance.kubevirt.io/v1beta1 kind: NodeMaintenance metadata: ... spec: nodeName: node-1.example.com reason: Node maintenance status: evictionPods: 31 pendingPods: - pod-example-workload-0 - httpd - httpd-manual phase: Running lastError: "Last failure message"2 totalpods: 5 ...
11.3. Resuming a node from maintenance mode Copiar enlaceEnlace copiado en el portapapeles!
Resuming a node brings it out of maintenance mode and makes it schedulable again.
Resume a node from maintenance mode from the web console, CLI, or by deleting the
NodeMaintenance
11.3.1. Resuming a node from maintenance mode in the web console Copiar enlaceEnlace copiado en el portapapeles!
Resume a node from maintenance mode using the Options menu
found on each node in the Compute → Nodes list, or using the Actions control of the Node Details screen.
Procedure
- In the OpenShift Virtualization console, click Compute → Nodes.
You can resume the node from this screen, which makes it easier to perform actions on multiple nodes in the one screen, or from the Node Details screen where you can view comprehensive details of the selected node:
-
Click the Options menu
at the end of the node and select Stop Maintenance.
- Click the node name to open the Node Details screen and click Actions → Stop Maintenance.
-
Click the Options menu
- Click Stop Maintenance in the confirmation window.
The node becomes schedulable, but virtual machine instances that were running on the node prior to maintenance will not automatically migrate back to this node.
11.3.2. Resuming a node from maintenance mode in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Resume a node from maintenance mode by making it schedulable again.
Procedure
Mark the node as schedulable. You can then resume scheduling new workloads on the node.
$ oc adm uncordon <node1>
11.3.3. Resuming a node from maintenance mode that was initiated with a NodeMaintenance CR Copiar enlaceEnlace copiado en el portapapeles!
You can resume a node by deleting the
NodeMaintenance
Prerequisites
-
Install the OpenShift Container Platform CLI .
oc -
Log in to the cluster as a user with privileges.
cluster-admin
Procedure
When your node maintenance task is complete, delete the active
CR:NodeMaintenance$ oc delete -f nodemaintenance-cr.yamlExample output
nodemaintenance.nodemaintenance.kubevirt.io "maintenance-example" deleted
11.4. Automatic renewal of TLS certificates Copiar enlaceEnlace copiado en el portapapeles!
All TLS certificates for OpenShift Virtualization components are renewed and rotated automatically. You are not required to refresh them manually.
11.4.1. TLS certificates automatic renewal schedules Copiar enlaceEnlace copiado en el portapapeles!
TLS certificates are automatically deleted and replaced according to the following schedule:
- KubeVirt certificates are renewed daily.
- Containerized Data Importer controller (CDI) certificates are renewed every 15 days.
- MAC pool certificates are renewed every year.
Automatic TLS certificate rotation does not disrupt any operations. For example, the following operations continue to function without any disruption:
- Migrations
- Image uploads
- VNC and console connections
11.5. Managing node labeling for obsolete CPU models Copiar enlaceEnlace copiado en el portapapeles!
You can schedule a virtual machine (VM) on a node as long as the VM CPU model and policy are supported by the node.
11.5.1. About node labeling for obsolete CPU models Copiar enlaceEnlace copiado en el portapapeles!
The OpenShift Virtualization Operator uses a predefined list of obsolete CPU models to ensure that a node supports only valid CPU models for scheduled VMs.
By default, the following CPU models are eliminated from the list of labels generated for the node:
Example 11.1. Obsolete CPU models
"486"
Conroe
athlon
core2duo
coreduo
kvm32
kvm64
n270
pentium
pentium2
pentium3
pentiumpro
phenom
qemu32
qemu64
This predefined list is not visible in the
HyperConverged
spec.obsoleteCPUs.cpuModels
HyperConverged
11.5.2. About node labeling for CPU features Copiar enlaceEnlace copiado en el portapapeles!
Through the process of iteration, the base CPU features in the minimum CPU model are eliminated from the list of labels generated for the node.
For example:
-
An environment might have two supported CPU models: and
Penryn.Haswell If
is specified as the CPU model forPenryn, each base CPU feature forminCPUis compared to the list of CPU features supported byPenryn.HaswellExample 11.2. CPU features supported by
Penrynapic clflush cmov cx16 cx8 de fpu fxsr lahf_lm lm mca mce mmx msr mtrr nx pae pat pge pni pse pse36 sep sse sse2 sse4.1 ssse3 syscall tscExample 11.3. CPU features supported by
Haswellaes apic avx avx2 bmi1 bmi2 clflush cmov cx16 cx8 de erms fma fpu fsgsbase fxsr hle invpcid lahf_lm lm mca mce mmx movbe msr mtrr nx pae pat pcid pclmuldq pge pni popcnt pse pse36 rdtscp rtm sep smep sse sse2 sse4.1 sse4.2 ssse3 syscall tsc tsc-deadline x2apic xsaveIf both
andPenrynsupport a specific CPU feature, a label is not created for that feature. Labels are generated for CPU features that are supported only byHaswelland not byHaswell.PenrynExample 11.4. Node labels created for CPU features after iteration
aes avx avx2 bmi1 bmi2 erms fma fsgsbase hle invpcid movbe pcid pclmuldq popcnt rdtscp rtm sse4.2 tsc-deadline x2apic xsave
11.5.3. Configuring obsolete CPU models Copiar enlaceEnlace copiado en el portapapeles!
You can configure a list of obsolete CPU models by editing the
HyperConverged
Procedure
Edit the
custom resource, specifying the obsolete CPU models in theHyperConvergedarray. For example:obsoleteCPUsapiVersion: hco.kubevirt.io/v1beta1 kind: HyperConverged metadata: name: kubevirt-hyperconverged namespace: openshift-cnv spec: obsoleteCPUs: cpuModels:1 - "<obsolete_cpu_1>" - "<obsolete_cpu_2>" minCPUModel: "<minimum_cpu_model>"2 - 1
- Replace the example values in the
cpuModelsarray with obsolete CPU models. Any value that you specify is added to a predefined list of obsolete CPU models. The predefined list is not visible in the CR. - 2
- Replace this value with the minimum CPU model that you want to use for basic CPU features. If you do not specify a value,
Penrynis used by default.
11.6. Preventing node reconciliation Copiar enlaceEnlace copiado en el portapapeles!
Use
skip-node
node-labeller
11.6.1. Using skip-node annotation Copiar enlaceEnlace copiado en el portapapeles!
If you want the
node-labeller
oc
Prerequisites
-
You have installed the OpenShift CLI ().
oc
Procedure
Annotate the node that you want to skip by running the following command:
$ oc annotate node <node_name> node-labeller.kubevirt.io/skip-node=true1 - 1
- Replace
<node_name>with the name of the relevant node to skip.
Reconciliation resumes on the next cycle after the node annotation is removed or set to false.
Chapter 12. Node networking Copiar enlaceEnlace copiado en el portapapeles!
12.1. Observing node network state Copiar enlaceEnlace copiado en el portapapeles!
Node network state is the network configuration for all nodes in the cluster.
12.1.1. About nmstate Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization uses nmstate to report on and configure the state of the node network. This makes it possible to modify network policy configuration, such as by creating a Linux bridge on all nodes, by applying a single configuration manifest to the cluster.
Node networking is monitored and updated by the following objects:
NodeNetworkState- Reports the state of the network on that node.
NodeNetworkConfigurationPolicy-
Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a
NodeNetworkConfigurationPolicymanifest to the cluster. NodeNetworkConfigurationEnactment- Reports the network policies enacted upon each node.
OpenShift Virtualization supports the use of the following nmstate interface types:
- Linux Bridge
- VLAN
- Bond
- Ethernet
If your OpenShift Container Platform cluster uses OVN-Kubernetes as the default Container Network Interface (CNI) provider, you cannot attach a Linux bridge or bonding to the default interface of a host because of a change in the host network topology of OVN-Kubernetes. As a workaround, you can use a secondary network interface connected to your host, or switch to the OpenShift SDN default CNI provider.
12.1.2. Viewing the network state of a node Copiar enlaceEnlace copiado en el portapapeles!
A
NodeNetworkState
Procedure
List all the
objects in the cluster:NodeNetworkState$ oc get nnsInspect a
object to view the network on that node. The output in this example has been redacted for clarity:NodeNetworkState$ oc get nns node01 -o yamlExample output
apiVersion: nmstate.io/v1beta1 kind: NodeNetworkState metadata: name: node011 status: currentState:2 dns-resolver: ... interfaces: ... route-rules: ... routes: ... lastSuccessfulUpdateTime: "2020-01-31T12:14:00Z"3 - 1
- The name of the
NodeNetworkStateobject is taken from the node. - 2
- The
currentStatecontains the complete network configuration for the node, including DNS, interfaces, and routes. - 3
- Timestamp of the last successful update. This is updated periodically as long as the node is reachable and can be used to evalute the freshness of the report.
12.2. Updating node network configuration Copiar enlaceEnlace copiado en el portapapeles!
You can update the node network configuration, such as adding or removing interfaces from nodes, by applying
NodeNetworkConfigurationPolicy
12.2.1. About nmstate Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization uses nmstate to report on and configure the state of the node network. This makes it possible to modify network policy configuration, such as by creating a Linux bridge on all nodes, by applying a single configuration manifest to the cluster.
Node networking is monitored and updated by the following objects:
NodeNetworkState- Reports the state of the network on that node.
NodeNetworkConfigurationPolicy-
Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a
NodeNetworkConfigurationPolicymanifest to the cluster. NodeNetworkConfigurationEnactment- Reports the network policies enacted upon each node.
OpenShift Virtualization supports the use of the following nmstate interface types:
- Linux Bridge
- VLAN
- Bond
- Ethernet
If your OpenShift Container Platform cluster uses OVN-Kubernetes as the default Container Network Interface (CNI) provider, you cannot attach a Linux bridge or bonding to the default interface of a host because of a change in the host network topology of OVN-Kubernetes. As a workaround, you can use a secondary network interface connected to your host, or switch to the OpenShift SDN default CNI provider.
12.2.2. Creating an interface on nodes Copiar enlaceEnlace copiado en el portapapeles!
Create an interface on nodes in the cluster by applying a
NodeNetworkConfigurationPolicy
By default, the manifest applies to all nodes in the cluster. To add the interface to specific nodes, add the
spec: nodeSelector
<key>:<value>
Procedure
Create the
manifest. The following example configures a Linux bridge on all worker nodes:NodeNetworkConfigurationPolicyapiVersion: nmstate.io/v1beta1 kind: NodeNetworkConfigurationPolicy metadata: name: <br1-eth1-policy>1 spec: nodeSelector:2 node-role.kubernetes.io/worker: ""3 desiredState: interfaces: - name: br1 description: Linux bridge with eth1 as a port4 type: linux-bridge state: up ipv4: dhcp: true enabled: true bridge: options: stp: enabled: false port: - name: eth1- 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses the
node-role.kubernetes.io/worker: ""node selector to select all worker nodes in the cluster. - 4
- Optional: Human-readable description for the interface.
Create the node network policy:
$ oc apply -f <br1-eth1-policy.yaml>1 - 1
- File name of the node network configuration policy manifest.
Additional resources
12.2.3. Confirming node network policy updates on nodes Copiar enlaceEnlace copiado en el portapapeles!
A
NodeNetworkConfigurationPolicy
When you apply a node network policy, a
NodeNetworkConfigurationEnactment
Procedure
To confirm that a policy has been applied to the cluster, list the policies and their status:
$ oc get nncpOptional: If a policy is taking longer than expected to successfully configure, you can inspect the requested state and status conditions of a particular policy:
$ oc get nncp <policy> -o yamlOptional: If a policy is taking longer than expected to successfully configure on all nodes, you can list the status of the enactments on the cluster:
$ oc get nnceOptional: To view the configuration of a particular enactment, including any error reporting for a failed configuration:
$ oc get nnce <node>.<policy> -o yaml
12.2.4. Removing an interface from nodes Copiar enlaceEnlace copiado en el portapapeles!
You can remove an interface from one or more nodes in the cluster by editing the
NodeNetworkConfigurationPolicy
state
absent
Removing an interface from a node does not automatically restore the node network configuration to a previous state. If you want to restore the previous state, you will need to define that node network configuration in the policy.
If you remove a bridge or bonding interface, any node NICs in the cluster that were previously attached or subordinate to that bridge or bonding interface are placed in a
down
up
Deleting the node network policy that added an interface does not change the configuration of the policy on the node. Although a
NodeNetworkConfigurationPolicy
Similarly, removing an interface does not delete the policy.
Procedure
Update the
manifest used to create the interface. The following example removes a Linux bridge and configures theNodeNetworkConfigurationPolicyNIC with DHCP to avoid losing connectivity:eth1apiVersion: nmstate.io/v1beta1 kind: NodeNetworkConfigurationPolicy metadata: name: <br1-eth1-policy>1 spec: nodeSelector:2 node-role.kubernetes.io/worker: ""3 desiredState: interfaces: - name: br1 type: linux-bridge state: absent4 - name: eth15 type: ethernet6 state: up7 ipv4: dhcp: true8 enabled: true9 - 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses the
node-role.kubernetes.io/worker: ""node selector to select all worker nodes in the cluster. - 4
- Changing the state to
absentremoves the interface. - 5
- The name of the interface that is to be unattached from the bridge interface.
- 6
- The type of interface. This example creates an Ethernet networking interface.
- 7
- The requested state for the interface.
- 8
- Optional: If you do not use
dhcp, you can either set a static IP or leave the interface without an IP address. - 9
- Enables
ipv4in this example.
Update the policy on the node and remove the interface:
$ oc apply -f <br1-eth1-policy.yaml>1 - 1
- File name of the policy manifest.
12.2.5. Example policy configurations for different interfaces Copiar enlaceEnlace copiado en el portapapeles!
12.2.5.1. Example: Linux bridge interface node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
Create a Linux bridge interface on nodes in the cluster by applying a
NodeNetworkConfigurationPolicy
The following YAML file is an example of a manifest for a Linux bridge interface. It includes samples values that you must replace with your own information.
apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
name: br1-eth1-policy
spec:
nodeSelector:
kubernetes.io/hostname: <node01>
desiredState:
interfaces:
- name: br1
description: Linux bridge with eth1 as a port
type: linux-bridge
state: up
ipv4:
dhcp: true
enabled: true
bridge:
options:
stp:
enabled: false
port:
- name: eth1
- 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses a
hostnamenode selector. - 4
- Name of the interface.
- 5
- Optional: Human-readable description of the interface.
- 6
- The type of interface. This example creates a bridge.
- 7
- The requested state for the interface after creation.
- 8
- Optional: If you do not use
dhcp, you can either set a static IP or leave the interface without an IP address. - 9
- Enables
ipv4in this example. - 10
- Disables
stpin this example. - 11
- The node NIC to which the bridge attaches.
12.2.5.2. Example: VLAN interface node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
Create a VLAN interface on nodes in the cluster by applying a
NodeNetworkConfigurationPolicy
The following YAML file is an example of a manifest for a VLAN interface. It includes samples values that you must replace with your own information.
apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
name: vlan-eth1-policy
spec:
nodeSelector:
kubernetes.io/hostname: <node01>
desiredState:
interfaces:
- name: eth1.102
description: VLAN using eth1
type: vlan
state: up
vlan:
base-iface: eth1
id: 102
- 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses a
hostnamenode selector. - 4
- Name of the interface.
- 5
- Optional: Human-readable description of the interface.
- 6
- The type of interface. This example creates a VLAN.
- 7
- The requested state for the interface after creation.
- 8
- The node NIC to which the VLAN is attached.
- 9
- The VLAN tag.
12.2.5.3. Example: Bond interface node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
Create a bond interface on nodes in the cluster by applying a
NodeNetworkConfigurationPolicy
OpenShift Virtualization only supports the following bond modes:
-
mode=1 active-backup
-
mode=2 balance-xor
-
mode=4 802.3ad
-
mode=5 balance-tlb
- mode=6 balance-alb
The following YAML file is an example of a manifest for a bond interface. It includes samples values that you must replace with your own information.
apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
name: bond0-eth1-eth2-policy
spec:
nodeSelector:
kubernetes.io/hostname: <node01>
desiredState:
interfaces:
- name: bond0
description: Bond enslaving eth1 and eth2
type: bond
state: up
ipv4:
dhcp: true
enabled: true
link-aggregation:
mode: active-backup
options:
miimon: '140'
slaves:
- eth1
- eth2
mtu: 1450
- 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses a
hostnamenode selector. - 4
- Name of the interface.
- 5
- Optional: Human-readable description of the interface.
- 6
- The type of interface. This example creates a bond.
- 7
- The requested state for the interface after creation.
- 8
- Optional: If you do not use
dhcp, you can either set a static IP or leave the interface without an IP address. - 9
- Enables
ipv4in this example. - 10
- The driver mode for the bond. This example uses an active backup mode.
- 11
- Optional: This example uses miimon to inspect the bond link every 140ms.
- 12
- The subordinate node NICs in the bond.
- 13
- Optional: The maximum transmission unit (MTU) for the bond. If not specified, this value is set to
1500by default.
12.2.5.4. Example: Ethernet interface node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
Configure an Ethernet interface on nodes in the cluster by applying a
NodeNetworkConfigurationPolicy
The following YAML file is an example of a manifest for an Ethernet interface. It includes sample values that you must replace with your own information.
apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
name: eth1-policy
spec:
nodeSelector:
kubernetes.io/hostname: <node01>
desiredState:
interfaces:
- name: eth1
description: Configuring eth1 on node01
type: ethernet
state: up
ipv4:
dhcp: true
enabled: true
- 1
- Name of the policy.
- 2
- Optional: If you do not include the
nodeSelectorparameter, the policy applies to all nodes in the cluster. - 3
- This example uses a
hostnamenode selector. - 4
- Name of the interface.
- 5
- Optional: Human-readable description of the interface.
- 6
- The type of interface. This example creates an Ethernet networking interface.
- 7
- The requested state for the interface after creation.
- 8
- Optional: If you do not use
dhcp, you can either set a static IP or leave the interface without an IP address. - 9
- Enables
ipv4in this example.
12.2.5.5. Example: Multiple interfaces in the same node network configuration policy Copiar enlaceEnlace copiado en el portapapeles!
You can create multiple interfaces in the same node network configuration policy. These interfaces can reference each other, allowing you to build and deploy a network configuration by using a single policy manifest.
The following example snippet creates a bond that is named
bond10
br1
...
interfaces:
- name: bond10
description: Bonding eth2 and eth3 for Linux bridge
type: bond
state: up
link-aggregation:
slaves:
- eth2
- eth3
- name: br1
description: Linux bridge on bond
type: linux-bridge
state: up
bridge:
port:
- name: bond10
...
12.2.6. Examples: IP management Copiar enlaceEnlace copiado en el portapapeles!
The following example configuration snippets demonstrate different methods of IP management.
These examples use the
ethernet
12.2.6.1. Static Copiar enlaceEnlace copiado en el portapapeles!
The following snippet statically configures an IP address on the Ethernet interface:
...
interfaces:
- name: eth1
description: static IP on eth1
type: ethernet
state: up
ipv4:
dhcp: false
address:
- ip: 192.168.122.250
prefix-length: 24
enabled: true
...
- 1
- Replace this value with the static IP address for the interface.
12.2.6.2. No IP address Copiar enlaceEnlace copiado en el portapapeles!
The following snippet ensures that the interface has no IP address:
...
interfaces:
- name: eth1
description: No IP on eth1
type: ethernet
state: up
ipv4:
enabled: false
...
12.2.6.3. Dynamic host configuration Copiar enlaceEnlace copiado en el portapapeles!
The following snippet configures an Ethernet interface that uses a dynamic IP address, gateway address, and DNS:
...
interfaces:
- name: eth1
description: DHCP on eth1
type: ethernet
state: up
ipv4:
dhcp: true
enabled: true
...
The following snippet configures an Ethernet interface that uses a dynamic IP address but does not use a dynamic gateway address or DNS:
...
interfaces:
- name: eth1
description: DHCP without gateway or DNS on eth1
type: ethernet
state: up
ipv4:
dhcp: true
auto-gateway: false
auto-dns: false
enabled: true
...
12.2.6.4. DNS Copiar enlaceEnlace copiado en el portapapeles!
The following snippet sets DNS configuration on the host.
...
interfaces:
...
dns-resolver:
config:
search:
- example.com
- example.org
server:
- 8.8.8.8
...
12.2.6.5. Static routing Copiar enlaceEnlace copiado en el portapapeles!
The following snippet configures a static route and a static IP on interface
eth1
...
interfaces:
- name: eth1
description: Static routing on eth1
type: ethernet
state: up
ipv4:
dhcp: false
address:
- ip: 192.0.2.251
prefix-length: 24
enabled: true
routes:
config:
- destination: 198.51.100.0/24
metric: 150
next-hop-address: 192.0.2.1
next-hop-interface: eth1
table-id: 254
...
12.3. Troubleshooting node network configuration Copiar enlaceEnlace copiado en el portapapeles!
If the node network configuration encounters an issue, the policy is automatically rolled back and the enactments report failure. This includes issues such as:
- The configuration fails to be applied on the host.
- The host loses connection to the default gateway.
- The host loses connection to the API server.
12.3.1. Troubleshooting an incorrect node network configuration policy configuration Copiar enlaceEnlace copiado en el portapapeles!
You can apply changes to the node network configuration across your entire cluster by applying a node network configuration policy. If you apply an incorrect configuration, you can use the following example to troubleshoot and correct the failed node network policy.
In this example, a Linux bridge policy is applied to an example cluster that has three control plane nodes (master) and three compute (worker) nodes. The policy fails to be applied because it references an incorrect interface. To find the error, investigate the available NMState resources. You can then update the policy with the correct configuration.
Procedure
Create a policy and apply it to your cluster. The following example creates a simple bridge on the
interface:ens01apiVersion: nmstate.io/v1beta1 kind: NodeNetworkConfigurationPolicy metadata: name: ens01-bridge-testfail spec: desiredState: interfaces: - name: br1 description: Linux bridge with the wrong port type: linux-bridge state: up ipv4: dhcp: true enabled: true bridge: options: stp: enabled: false port: - name: ens01$ oc apply -f ens01-bridge-testfail.yamlExample output
nodenetworkconfigurationpolicy.nmstate.io/ens01-bridge-testfail createdVerify the status of the policy by running the following command:
$ oc get nncpThe output shows that the policy failed:
Example output
NAME STATUS ens01-bridge-testfail FailedToConfigureHowever, the policy status alone does not indicate if it failed on all nodes or a subset of nodes.
List the node network configuration enactments to see if the policy was successful on any of the nodes. If the policy failed for only a subset of nodes, it suggests that the problem is with a specific node configuration. If the policy failed on all nodes, it suggests that the problem is with the policy.
$ oc get nnceThe output shows that the policy failed on all nodes:
Example output
NAME STATUS control-plane-1.ens01-bridge-testfail FailedToConfigure control-plane-2.ens01-bridge-testfail FailedToConfigure control-plane-3.ens01-bridge-testfail FailedToConfigure compute-1.ens01-bridge-testfail FailedToConfigure compute-2.ens01-bridge-testfail FailedToConfigure compute-3.ens01-bridge-testfail FailedToConfigureView one of the failed enactments and look at the traceback. The following command uses the output tool
to filter the output:jsonpath$ oc get nnce compute-1.ens01-bridge-testfail -o jsonpath='{.status.conditions[?(@.type=="Failing")].message}'This command returns a large traceback that has been edited for brevity:
Example output
error reconciling NodeNetworkConfigurationPolicy at desired state apply: , failed to execute nmstatectl set --no-commit --timeout 480: 'exit status 1' '' ... libnmstate.error.NmstateVerificationError: desired ======= --- name: br1 type: linux-bridge state: up bridge: options: group-forward-mask: 0 mac-ageing-time: 300 multicast-snooping: true stp: enabled: false forward-delay: 15 hello-time: 2 max-age: 20 priority: 32768 port: - name: ens01 description: Linux bridge with the wrong port ipv4: address: [] auto-dns: true auto-gateway: true auto-routes: true dhcp: true enabled: true ipv6: enabled: false mac-address: 01-23-45-67-89-AB mtu: 1500 current ======= --- name: br1 type: linux-bridge state: up bridge: options: group-forward-mask: 0 mac-ageing-time: 300 multicast-snooping: true stp: enabled: false forward-delay: 15 hello-time: 2 max-age: 20 priority: 32768 port: [] description: Linux bridge with the wrong port ipv4: address: [] auto-dns: true auto-gateway: true auto-routes: true dhcp: true enabled: true ipv6: enabled: false mac-address: 01-23-45-67-89-AB mtu: 1500 difference ========== --- desired +++ current @@ -13,8 +13,7 @@ hello-time: 2 max-age: 20 priority: 32768 - port: - - name: ens01 + port: [] description: Linux bridge with the wrong port ipv4: address: [] line 651, in _assert_interfaces_equal\n current_state.interfaces[ifname],\nlibnmstate.error.NmstateVerificationError:The
lists theNmstateVerificationErrorpolicy configuration, thedesiredconfiguration of the policy on the node, and thecurrenthighlighting the parameters that do not match. In this example, thedifferenceis included in theport, which suggests that the problem is the port configuration in the policy.differenceTo ensure that the policy is configured properly, view the network configuration for one or all of the nodes by requesting the
object. The following command returns the network configuration for theNodeNetworkStatenode:control-plane-1$ oc get nns control-plane-1 -o yamlThe output shows that the interface name on the nodes is
but the failed policy incorrectly usesens1:ens01Example output
- ipv4: ... name: ens1 state: up type: ethernetCorrect the error by editing the existing policy:
$ oc edit nncp ens01-bridge-testfail... port: - name: ens1Save the policy to apply the correction.
Check the status of the policy to ensure it updated successfully:
$ oc get nncpExample output
NAME STATUS ens01-bridge-testfail SuccessfullyConfigured
The updated policy is successfully configured on all nodes in the cluster.
Chapter 13. Logging, events, and monitoring Copiar enlaceEnlace copiado en el portapapeles!
13.1. Viewing virtual machine logs Copiar enlaceEnlace copiado en el portapapeles!
13.1.1. Understanding virtual machine logs Copiar enlaceEnlace copiado en el portapapeles!
Logs are collected for OpenShift Container Platform builds, deployments, and pods. In OpenShift Virtualization, virtual machine logs can be retrieved from the virtual machine launcher pod in either the web console or the CLI.
The
-f
If the launcher pod is failing to start, use the
--previous
ErrImagePull
ImagePullBackOff
13.1.2. Viewing virtual machine logs in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Get virtual machine logs from the virtual machine launcher pod.
Procedure
Use the following command:
$ oc logs <virt-launcher-name>
13.1.3. Viewing virtual machine logs in the web console Copiar enlaceEnlace copiado en el portapapeles!
Get virtual machine logs from the associated virtual machine launcher pod.
Procedure
- In the OpenShift Virtualization console, click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
-
In the Details tab, click the pod in the Pod section.
virt-launcher-<name> - Click Logs.
13.2. Viewing events Copiar enlaceEnlace copiado en el portapapeles!
13.2.1. Understanding virtual machine events Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Container Platform events are records of important life-cycle information in a namespace and are useful for monitoring and troubleshooting resource scheduling, creation, and deletion issues.
OpenShift Virtualization adds events for virtual machines and virtual machine instances. These can be viewed from either the web console or the CLI.
See also: Viewing system event information in an OpenShift Container Platform cluster.
13.2.2. Viewing the events for a virtual machine in the web console Copiar enlaceEnlace copiado en el portapapeles!
You can view the stream events for a running a virtual machine from the Virtual Machine Overview panel of the web console.
The ▮▮ button pauses the events stream.
The ▶ button continues a paused events stream.
Procedure
- Click Workloads → Virtualization from the side menu.
- Click the Virtual Machines tab.
- Select a virtual machine to open the Virtual Machine Overview screen.
- Click Events to view all events for the virtual machine.
13.2.3. Viewing namespace events in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Use the OpenShift Container Platform client to get the events for a namespace.
Procedure
In the namespace, use the
command:oc get$ oc get events
13.2.4. Viewing resource events in the CLI Copiar enlaceEnlace copiado en el portapapeles!
Events are included in the resource description, which you can get using the OpenShift Container Platform client.
Procedure
In the namespace, use the
command. The following example shows how to get the events for a virtual machine, a virtual machine instance, and the virt-launcher pod for a virtual machine:oc describe$ oc describe vm <vm>$ oc describe vmi <vmi>$ oc describe pod virt-launcher-<name>
13.3. Diagnosing data volumes using events and conditions Copiar enlaceEnlace copiado en el portapapeles!
Use the
oc describe
13.3.1. About conditions and events Copiar enlaceEnlace copiado en el portapapeles!
Diagnose data volume issues by examining the output of the
Conditions
Events
$ oc describe dv <DataVolume>
There are three
Types
Conditions
-
Bound -
Running -
Ready
The
Events
-
of event
Type -
for logging
Reason -
of the event
Source -
containing additional diagnostic information.
Message
The output from
oc describe
Events
An event is generated when either
Status
Reason
Message
For example, if you misspell the URL during an import operation, the import generates a 404 message. That message change generates an event with a reason. The output in the
Conditions
13.3.2. Analyzing data volumes using conditions and events Copiar enlaceEnlace copiado en el portapapeles!
By inspecting the
Conditions
Events
describe
There are many different combinations of conditions. Each must be evaluated in its unique context.
Examples of various combinations follow.
- – A successfully bound PVC displays in this example.
BoundNote that the
isType, so theBoundisStatus. If the PVC is not bound, theTrueisStatus.FalseWhen the PVC is bound, an event is generated stating that the PVC is bound. In this case, the
isReasonandBoundisStatus. TheTrueindicates which PVC owns the data volume.Message, in theMessagesection, provides further details including how long the PVC has been bound (Events) and by what resource (Age), in this caseFrom:datavolume-controllerExample output
Status: Conditions: Last Heart Beat Time: 2020-07-15T03:58:24Z Last Transition Time: 2020-07-15T03:58:24Z Message: PVC win10-rootdisk Bound Reason: Bound Status: True Type: Bound Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Bound 24s datavolume-controller PVC example-dv Bound - – In this case, note that
RunningisTypeandRunningisStatus, indicating that an event has occurred that caused an attempted operation to fail, changing the Status fromFalsetoTrue.FalseHowever, note that
isReasonand theCompletedfield indicatesMessage.Import CompleteIn the
section, theEventsandReasoncontain additional troubleshooting information about the failed operation. In this example, theMessagedisplays an inability to connect due to aMessage, listed in the404section’s firstEvents.WarningFrom this information, you conclude that an import operation was running, creating contention for other operations that are attempting to access the data volume:
Example output
Status: Conditions: Last Heart Beat Time: 2020-07-15T04:31:39Z Last Transition Time: 2020-07-15T04:31:39Z Message: Import Complete Reason: Completed Status: False Type: Running Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Error 12s (x2 over 14s) datavolume-controller Unable to connect to http data source: expected status code 200, got 404. Status: 404 Not Found - – If
ReadyisTypeandReadyisStatus, then the data volume is ready to be used, as in the following example. If the data volume is not ready to be used, theTrueisStatus:FalseExample output
Status: Conditions: Last Heart Beat Time: 2020-07-15T04:31:39Z Last Transition Time: 2020-07-15T04:31:39Z Status: True Type: Ready
13.4. Viewing information about virtual machine workloads Copiar enlaceEnlace copiado en el portapapeles!
You can view high-level information about your virtual machines by using the Virtual Machines dashboard in the OpenShift Container Platform web console.
13.4.1. About the Virtual Machines dashboard Copiar enlaceEnlace copiado en el portapapeles!
Access virtual machines from the OpenShift Container Platform web console by navigating to the Workloads → Virtualization page. The Workloads → Virtualization page contains two tabs:
- Virtual Machines
- Virtual Machine Templates
The following cards describe each virtual machine:
Details provides identifying information about the virtual machine, including:
- Name
- Namespace
- Date of creation
- Node name
- IP address
Inventory lists the virtual machine’s resources, including:
- Network interface controllers (NICs)
- Disks
Status includes:
- The current status of the virtual machine
- A note indicating whether or not the QEMU guest agent is installed on the virtual machine
Utilization includes charts that display usage data for:
- CPU
- Memory
- Filesystem
- Network transfer
Use the drop-down list to choose a duration for the utilization data. The available options are 1 Hour, 6 Hours, and 24 Hours.
- Events lists messages about virtual machine activity over the past hour. To view additional events, click View all.
13.5. Monitoring virtual machine health Copiar enlaceEnlace copiado en el portapapeles!
A virtual machine instance (VMI) can become unhealthy due to transient issues such as connectivity loss, deadlocks, or problems with external dependencies. A health check periodically performs diagnostics on a VMI by using any combination of the readiness and liveness probes.
13.5.1. About readiness and liveness probes Copiar enlaceEnlace copiado en el portapapeles!
Use readiness and liveness probes to detect and handle unhealthy virtual machine instances (VMIs). You can include one or more probes in the specification of the VMI to ensure that traffic does not reach a VMI that is not ready for it and that a new instance is created when a VMI becomes unresponsive.
A readiness probe determines whether a VMI is ready to accept service requests. If the probe fails, the VMI is removed from the list of available endpoints until the VMI is ready.
A liveness probe determines whether a VMI is responsive. If the probe fails, the VMI is deleted and a new instance is created to restore responsiveness.
You can configure readiness and liveness probes by setting the
spec.readinessProbe
spec.livenessProbe
VirtualMachineInstance
- HTTP GET
- The probe determines the health of the VMI by using a web hook. The test is successful if the HTTP response code is between 200 and 399. You can use an HTTP GET test with applications that return HTTP status codes when they are completely initialized.
- TCP socket
- The probe attempts to open a socket to the VMI. The VMI is only considered healthy if the probe can establish a connection. You can use a TCP socket test with applications that do not start listening until initialization is complete.
13.5.2. Defining an HTTP readiness probe Copiar enlaceEnlace copiado en el portapapeles!
Define an HTTP readiness probe by setting the
spec.readinessProbe.httpGet
Procedure
Include details of the readiness probe in the VMI configuration file.
Sample readiness probe with an HTTP GET test
# ... spec: readinessProbe: httpGet:1 port: 15002 path: /healthz3 httpHeaders: - name: Custom-Header value: Awesome initialDelaySeconds: 1204 periodSeconds: 205 timeoutSeconds: 106 failureThreshold: 37 successThreshold: 38 # ...- 1
- The HTTP GET request to perform to connect to the VMI.
- 2
- The port of the VMI that the probe queries. In the above example, the probe queries port 1500.
- 3
- The path to access on the HTTP server. In the above example, if the handler for the server’s /healthz path returns a success code, the VMI is considered to be healthy. If the handler returns a failure code, the VMI is removed from the list of available endpoints.
- 4
- The time, in seconds, after the VMI starts before the readiness probe is initiated.
- 5
- The delay, in seconds, between performing probes. The default delay is 10 seconds. This value must be greater than
timeoutSeconds. - 6
- The number of seconds of inactivity after which the probe times out and the VMI is assumed to have failed. The default value is 1. This value must be lower than
periodSeconds. - 7
- The number of times that the probe is allowed to fail. The default is 3. After the specified number of attempts, the pod is marked
Unready. - 8
- The number of times that the probe must report success, after a failure, to be considered successful. The default is 1.
Create the VMI by running the following command:
$ oc create -f <file_name>.yaml
13.5.3. Defining a TCP readiness probe Copiar enlaceEnlace copiado en el portapapeles!
Define a TCP readiness probe by setting the
spec.readinessProbe.tcpSocket
Procedure
Include details of the TCP readiness probe in the VMI configuration file.
Sample readiness probe with a TCP socket test
... spec: readinessProbe: initialDelaySeconds: 1201 periodSeconds: 202 tcpSocket:3 port: 15004 timeoutSeconds: 105 ...- 1
- The time, in seconds, after the VMI starts before the readiness probe is initiated.
- 2
- The delay, in seconds, between performing probes. The default delay is 10 seconds. This value must be greater than
timeoutSeconds. - 3
- The TCP action to perform.
- 4
- The port of the VMI that the probe queries.
- 5
- The number of seconds of inactivity after which the probe times out and the VMI is assumed to have failed. The default value is 1. This value must be lower than
periodSeconds.
Create the VMI by running the following command:
$ oc create -f <file_name>.yaml
13.5.4. Defining an HTTP liveness probe Copiar enlaceEnlace copiado en el portapapeles!
Define an HTTP liveness probe by setting the
spec.livenessProbe.httpGet
Procedure
Include details of the HTTP liveness probe in the VMI configuration file.
Sample liveness probe with an HTTP GET test
# ... spec: livenessProbe: initialDelaySeconds: 1201 periodSeconds: 202 httpGet:3 port: 15004 path: /healthz5 httpHeaders: - name: Custom-Header value: Awesome timeoutSeconds: 106 # ...- 1
- The time, in seconds, after the VMI starts before the liveness probe is initiated.
- 2
- The delay, in seconds, between performing probes. The default delay is 10 seconds. This value must be greater than
timeoutSeconds. - 3
- The HTTP GET request to perform to connect to the VMI.
- 4
- The port of the VMI that the probe queries. In the above example, the probe queries port 1500. The VMI installs and runs a minimal HTTP server on port 1500 via cloud-init.
- 5
- The path to access on the HTTP server. In the above example, if the handler for the server’s
/healthzpath returns a success code, the VMI is considered to be healthy. If the handler returns a failure code, the VMI is deleted and a new instance is created. - 6
- The number of seconds of inactivity after which the probe times out and the VMI is assumed to have failed. The default value is 1. This value must be lower than
periodSeconds.
Create the VMI by running the following command:
$ oc create -f <file_name>.yaml
13.5.5. Template: Virtual machine instance configuration file for defining health checks Copiar enlaceEnlace copiado en el portapapeles!
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:
labels:
special: vmi-fedora
name: vmi-fedora
spec:
domain:
devices:
disks:
- disk:
bus: virtio
name: containerdisk
- disk:
bus: virtio
name: cloudinitdisk
resources:
requests:
memory: 1024M
readinessProbe:
httpGet:
port: 1500
initialDelaySeconds: 120
periodSeconds: 20
timeoutSeconds: 10
failureThreshold: 3
successThreshold: 3
terminationGracePeriodSeconds: 0
volumes:
- name: containerdisk
containerDisk:
image: kubevirt/fedora-cloud-registry-disk-demo
- cloudInitNoCloud:
userData: |-
#cloud-config
password: fedora
chpasswd: { expire: False }
bootcmd:
- setenforce 0
- dnf install -y nmap-ncat
- systemd-run --unit=httpserver nc -klp 1500 -e '/usr/bin/echo -e HTTP/1.1 200 OK\\n\\nHello World!'
name: cloudinitdisk
13.6. Using the OpenShift Container Platform dashboard to get cluster information Copiar enlaceEnlace copiado en el portapapeles!
Access the OpenShift Container Platform dashboard, which captures high-level information about the cluster, by clicking Home > Dashboards > Overview from the OpenShift Container Platform web console.
The OpenShift Container Platform dashboard provides various cluster information, captured in individual dashboard cards.
13.6.1. About the OpenShift Container Platform dashboards page Copiar enlaceEnlace copiado en el portapapeles!
The OpenShift Container Platform dashboard consists of the following cards:
Details provides a brief overview of informational cluster details.
Status include ok, error, warning, in progress, and unknown. Resources can add custom status names.
- Cluster ID
- Provider
- Version
Cluster Inventory details number of resources and associated statuses. It is helpful when intervention is required to resolve problems, including information about:
- Number of nodes
- Number of pods
- Persistent storage volume claims
- Virtual machines (available if OpenShift Virtualization is installed)
- Bare metal hosts in the cluster, listed according to their state (only available in metal3 environment).
- Cluster Health summarizes the current health of the cluster as a whole, including relevant alerts and descriptions. If OpenShift Virtualization is installed, the overall health of OpenShift Virtualization is diagnosed as well. If more than one subsystem is present, click See All to view the status of each subsystem.
Cluster Capacity charts help administrators understand when additional resources are required in the cluster. The charts contain an inner ring that displays current consumption, while an outer ring displays thresholds configured for the resource, including information about:
- CPU time
- Memory allocation
- Storage consumed
- Network resources consumed
- Cluster Utilization shows the capacity of various resources over a specified period of time, to help administrators understand the scale and frequency of high resource consumption.
- Events lists messages related to recent activity in the cluster, such as pod creation or virtual machine migration to another host.
- Top Consumers helps administrators understand how cluster resources are consumed. Click on a resource to jump to a detailed page listing pods and nodes that consume the largest amount of the specified cluster resource (CPU, memory, or storage).
13.7. OpenShift Container Platform cluster monitoring, logging, and Telemetry Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Container Platform provides various resources for monitoring at the cluster level.
13.7.1. About OpenShift Container Platform monitoring Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Container Platform includes a pre-configured, pre-installed, and self-updating monitoring stack that provides monitoring for core platform components. OpenShift Container Platform delivers monitoring best practices out of the box. A set of alerts are included by default that immediately notify cluster administrators about issues with a cluster. Default dashboards in the OpenShift Container Platform web console include visual representations of cluster metrics to help you to quickly understand the state of your cluster.
After installing OpenShift Container Platform 4.8, cluster administrators can optionally enable monitoring for user-defined projects. By using this feature, cluster administrators, developers, and other users can specify how services and pods are monitored in their own projects. You can then query metrics, review dashboards, and manage alerting rules and silences for your own projects in the OpenShift Container Platform web console.
Cluster administrators can grant developers and other users permission to monitor their own projects. Privileges are granted by assigning one of the predefined monitoring roles.
13.7.2. About OpenShift Logging components Copiar enlaceEnlace copiado en el portapapeles!
The OpenShift Logging components include a collector deployed to each node in the OpenShift Container Platform cluster that collects all node and container logs and writes them to a log store. You can use a centralized web UI to create rich visualizations and dashboards with the aggregated data.
The major components of OpenShift Logging are:
- collection - This is the component that collects logs from the cluster, formats them, and forwards them to the log store. The current implementation is Fluentd.
- log store - This is where the logs are stored. The default implementation is Elasticsearch. You can use the default Elasticsearch log store or forward logs to external log stores. The default log store is optimized and tested for short-term storage.
- visualization - This is the UI component you can use to view logs, graphs, charts, and so forth. The current implementation is Kibana.
For more information on OpenShift Logging, see the OpenShift Logging documentation.
13.7.3. About Telemetry Copiar enlaceEnlace copiado en el portapapeles!
Telemetry sends a carefully chosen subset of the cluster monitoring metrics to Red Hat. The Telemeter Client fetches the metrics values every four minutes and thirty seconds and uploads the data to Red Hat. These metrics are described in this document.
This stream of data is used by Red Hat to monitor the clusters in real-time and to react as necessary to problems that impact our customers. It also allows Red Hat to roll out OpenShift Container Platform upgrades to customers to minimize service impact and continuously improve the upgrade experience.
This debugging information is available to Red Hat Support and Engineering teams with the same restrictions as accessing data reported through support cases. All connected cluster information is used by Red Hat to help make OpenShift Container Platform better and more intuitive to use.
13.7.3.1. Information collected by Telemetry Copiar enlaceEnlace copiado en el portapapeles!
The following information is collected by Telemetry:
- The unique random identifier that is generated during an installation
- Version information, including the OpenShift Container Platform cluster version and installed update details that are used to determine update version availability
- Update information, including the number of updates available per cluster, the channel and image repository used for an update, update progress information, and the number of errors that occur in an update
- The name of the provider platform that OpenShift Container Platform is deployed on and the data center location
- Sizing information about clusters, machine types, and machines, including the number of CPU cores and the amount of RAM used for each
- The number of running virtual machine instances in a cluster
- The number of etcd members and the number of objects stored in the etcd cluster
- The OpenShift Container Platform framework components installed in a cluster and their condition and status
- Usage information about components, features, and extensions
- Usage details about Technology Previews and unsupported configurations
- Information about degraded software
-
Information about nodes that are marked as
NotReady - Events for all namespaces listed as "related objects" for a degraded Operator
- Configuration details that help Red Hat Support to provide beneficial support for customers, including node configuration at the cloud infrastructure level, hostnames, IP addresses, Kubernetes pod names, namespaces, and services
- Information about the validity of certificates
- Number of application builds by build strategy type
Telemetry does not collect identifying information such as user names or passwords. Red Hat does not intend to collect personal information. If Red Hat discovers that personal information has been inadvertently received, Red Hat will delete such information. To the extent that any telemetry data constitutes personal data, please refer to the Red Hat Privacy Statement for more information about Red Hat’s privacy practices.
13.7.4. CLI troubleshooting and debugging commands Copiar enlaceEnlace copiado en el portapapeles!
For a list of the
oc
13.8. Prometheus queries for virtual resources Copiar enlaceEnlace copiado en el portapapeles!
OpenShift Virtualization provides metrics for monitoring how infrastructure resources are consumed in the cluster. The metrics cover the following resources:
- vCPU
- Network
- Storage
- Guest memory swapping
Use the OpenShift Container Platform monitoring dashboard to query virtualization metrics.
13.8.1. Prerequisites Copiar enlaceEnlace copiado en el portapapeles!
-
To use the vCPU metric, the kernel argument must be applied to the
schedstats=enableobject. This kernel argument enables scheduler statistics used for debugging and performance tuning and adds a minor additional load to the scheduler. See the OpenShift Container Platform machine configuration tasks documentation for more information on applying a kernel argument.MachineConfig - For guest memory swapping queries to return data, memory swapping must be enabled on the virtual guests.
13.8.2. Querying metrics Copiar enlaceEnlace copiado en el portapapeles!
The OpenShift Container Platform monitoring dashboard enables you to run Prometheus Query Language (PromQL) queries to examine metrics visualized on a plot. This functionality provides information about the state of a cluster and any user-defined workloads that you are monitoring.
As a cluster administrator, you can query metrics for all core OpenShift Container Platform and user-defined projects.
As a developer, you must specify a project name when querying metrics. You must have the required privileges to view metrics for the selected project.
13.8.2.1. Querying metrics for all projects as a cluster administrator Copiar enlaceEnlace copiado en el portapapeles!
As a cluster administrator or as a user with view permissions for all projects, you can access metrics for all default OpenShift Container Platform and user-defined projects in the Metrics UI.
Only cluster administrators have access to the third-party UIs provided with OpenShift Container Platform Monitoring.
Prerequisites
-
You have access to the cluster as a user with the role or with view permissions for all projects.
cluster-admin -
You have installed the OpenShift CLI ().
oc
Procedure
- In the Administrator perspective within the OpenShift Container Platform web console, select Monitoring → Metrics.
- Select Insert Metric at Cursor to view a list of predefined queries.
- To create a custom query, add your Prometheus Query Language (PromQL) query to the Expression field.
- To add multiple queries, select Add Query.
-
To delete a query, select
next to the query, then choose Delete query.
-
To disable a query from being run, select
next to the query and choose Disable query.
Select Run Queries to run the queries that you have created. The metrics from the queries are visualized on the plot. If a query is invalid, the UI shows an error message.
NoteQueries that operate on large amounts of data might time out or overload the browser when drawing time series graphs. To avoid this, select Hide graph and calibrate your query using only the metrics table. Then, after finding a feasible query, enable the plot to draw the graphs.
- Optional: The page URL now contains the queries you ran. To use this set of queries again in the future, save this URL.
13.8.2.2. Querying metrics for user-defined projects as a developer Copiar enlaceEnlace copiado en el portapapeles!
You can access metrics for a user-defined project as a developer or as a user with view permissions for the project.
In the Developer perspective, the Metrics UI includes some predefined CPU, memory, bandwidth, and network packet queries for the selected project. You can also run custom Prometheus Query Language (PromQL) queries for CPU, memory, bandwidth, network packet and application metrics for the project.
Developers can only use the Developer perspective and not the Administrator perspective. As a developer, you can only query metrics for one project at a time. Developers cannot access the third-party UIs provided with OpenShift Container Platform monitoring that are for core platform components. Instead, use the Metrics UI for your user-defined project.
Prerequisites
- You have access to the cluster as a developer or as a user with view permissions for the project that you are viewing metrics for.
- You have enabled monitoring for user-defined projects.
- You have deployed a service in a user-defined project.
-
You have created a custom resource definition (CRD) for the service to define how the service is monitored.
ServiceMonitor
Procedure
- From the Developer perspective in the OpenShift Container Platform web console, select Monitoring → Metrics.
- Select the project that you want to view metrics for in the Project: list.
Choose a query from the Select Query list, or run a custom PromQL query by selecting Show PromQL.
NoteIn the Developer perspective, you can only run one query at a time.
13.8.3. Virtualization metrics Copiar enlaceEnlace copiado en el portapapeles!
The following metric descriptions include example Prometheus Query Language (PromQL) queries. These metrics are not an API and might change between versions.
The following examples use
topk
13.8.3.1. vCPU metrics Copiar enlaceEnlace copiado en el portapapeles!
The following query can identify virtual machines that are waiting for Input/Output (I/O):
kubevirt_vmi_vcpu_wait_seconds- Returns the wait time (in seconds) for a virtual machine’s vCPU.
A value above '0' means that the vCPU wants to run, but the host scheduler cannot run it yet. This inability to run indicates that there is an issue with I/O.
To query the vCPU metric, the
schedstats=enable
MachineConfig
Example vCPU wait time query
topk(3, sum by (name, namespace) (rate(kubevirt_vmi_vcpu_wait_seconds[6m]))) > 0
- 1
- This query returns the top 3 VMs waiting for I/O at every given moment over a six-minute time period.
13.8.3.2. Network metrics Copiar enlaceEnlace copiado en el portapapeles!
The following queries can identify virtual machines that are saturating the network:
kubevirt_vmi_network_receive_bytes_total- Returns the total amount of traffic received (in bytes) on the virtual machine’s network.
kubevirt_vmi_network_transmit_bytes_total- Returns the total amount of traffic transmitted (in bytes) on the virtual machine’s network.
Example network traffic query
topk(3, sum by (name, namespace) (rate(kubevirt_vmi_network_receive_bytes_total[6m])) + sum by (name, namespace) (rate(kubevirt_vmi_network_transmit_bytes_total[6m]))) > 0
- 1
- This query returns the top 3 VMs transmitting the most network traffic at every given moment over a six-minute time period.
13.8.3.3. Storage metrics Copiar enlaceEnlace copiado en el portapapeles!
13.8.3.3.1. Storage-related traffic Copiar enlaceEnlace copiado en el portapapeles!
The following queries can identify VMs that are writing large amounts of data:
kubevirt_vmi_storage_read_traffic_bytes_total- Returns the total amount (in bytes) of the virtual machine’s storage-related traffic.
kubevirt_vmi_storage_write_traffic_bytes_total- Returns the total amount of storage writes (in bytes) of the virtual machine’s storage-related traffic.
Example storage-related traffic query
topk(3, sum by (name, namespace) (rate(kubevirt_vmi_storage_read_traffic_bytes_total[6m])) + sum by (name, namespace) (rate(kubevirt_vmi_storage_write_traffic_bytes_total[6m]))) > 0
- 1
- This query returns the top 3 VMs performing the most storage traffic at every given moment over a six-minute time period.
13.8.3.3.2. I/O performance Copiar enlaceEnlace copiado en el portapapeles!
The following queries can determine the I/O performance of storage devices:
kubevirt_vmi_storage_iops_read_total- Returns the amount of write I/O operations the virtual machine is performing per second.
kubevirt_vmi_storage_iops_write_total- Returns the amount of read I/O operations the virtual machine is performing per second.
Example I/O performance query
topk(3, sum by (name, namespace) (rate(kubevirt_vmi_storage_iops_read_total[6m])) + sum by (name, namespace) (rate(kubevirt_vmi_storage_iops_write_total[6m]))) > 0
- 1
- This query returns the top 3 VMs performing the most I/O operations per second at every given moment over a six-minute time period.
13.8.3.4. Guest memory swapping metrics Copiar enlaceEnlace copiado en el portapapeles!
The following queries can identify which swap-enabled guests are performing the most memory swapping:
kubevirt_vmi_memory_swap_in_traffic_bytes_total- Returns the total amount (in bytes) of memory the virtual guest is swapping in.
kubevirt_vmi_memory_swap_out_traffic_bytes_total- Returns the total amount (in bytes) of memory the virtual guest is swapping out.
Example memory swapping query
topk(3, sum by (name, namespace) (rate(kubevirt_vmi_memory_swap_in_traffic_bytes_total[6m])) + sum by (name, namespace) (rate(kubevirt_vmi_memory_swap_out_traffic_bytes_total[6m]))) > 0
- 1
- This query returns the top 3 VMs where the guest is performing the most memory swapping at every given moment over a six-minute time period.
Memory swapping indicates that the virtual machine is under memory pressure. Increasing the memory allocation of the virtual machine can mitigate this issue.
13.9. Collecting data for Red Hat Support Copiar enlaceEnlace copiado en el portapapeles!
When you submit a support case to Red Hat Support, it is helpful to provide debugging information for OpenShift Container Platform and OpenShift Virtualization by using the following tools:
- must-gather tool
-
The
must-gathertool collects diagnostic information, including resource definitions and service logs. - Prometheus
- Prometheus is a time-series database and a rule evaluation engine for metrics. Prometheus sends alerts to Alertmanager for processing.
- Alertmanager
- The Alertmanager service handles alerts received from Prometheus. The Alertmanager is also responsible for sending the alerts to external notification systems.
13.9.1. Collecting data about your environment Copiar enlaceEnlace copiado en el portapapeles!
Collecting data about your environment minimizes the time required to analyze and determine the root cause.
Prerequisites
- Set the retention time for Prometheus metrics data to a minimum of seven days.
- Configure the Alertmanager to capture relevant alerts and to send them to a dedicated mailbox so that they can be viewed and persisted outside the cluster.
- Record the exact number of affected nodes and virtual machines.
Procedure
-
Collect data for the cluster by using the default
must-gatherimage.must-gather -
Collect data for Red Hat OpenShift Container Storage, if necessary.
must-gather -
Collect data for OpenShift Virtualization by using the OpenShift Virtualization
must-gatherimage.must-gather - Collect Prometheus metrics for the cluster.
13.9.2. Collecting data about virtual machines Copiar enlaceEnlace copiado en el portapapeles!
Collecting data about malfunctioning virtual machines (VMs) minimizes the time required to analyze and determine the root cause.
Prerequisites
Windows VMs:
- Record the Windows patch update details for Red Hat Support.
- Install the latest version of the VirtIO drivers. The VirtIO drivers include the QEMU guest agent.
- If Remote Desktop Protocol (RDP) is enabled, try to connect to the VMs with RDP to determine whether there is a problem with the connection software.
Procedure
-
Collect detailed data about the malfunctioning VMs.
must-gather - Collect screenshots of VMs that have crashed before you restart them.
- Record factors that the malfunctioning VMs have in common. For example, the VMs have the same host or network.
13.9.3. Using the must-gather tool for OpenShift Virtualization Copiar enlaceEnlace copiado en el portapapeles!
You can collect data about OpenShift Virtualization resources by running the
must-gather
The default data collection includes information about the following resources:
- OpenShift Virtualization Operator namespaces, including child objects
- OpenShift Virtualization custom resource definitions
- Namespaces that contain virtual machines
- Basic virtual machine definitions
Procedure
Run the following command to collect data about OpenShift Virtualization:
$ oc adm must-gather --image-stream=openshift/must-gather \ --image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8:v{HCOVersion}
13.9.3.1. must-gather tool options Copiar enlaceEnlace copiado en el portapapeles!
You can specify a combination of scripts and environment variables for the following options:
- Collecting detailed virtual machine (VM) information from a namespace
- Collecting detailed information about specified VMs
- Collecting image and image stream information
-
Limiting the maximum number of parallel processes used by the tool
must-gather
13.9.3.1.1. Parameters Copiar enlaceEnlace copiado en el portapapeles!
Environment variables
You can specify environment variables for a compatible script.
NS=<namespace_name>-
Collect virtual machine information, including
virt-launcherpod details, from the namespace that you specify. TheVirtualMachineandVirtualMachineInstanceCR data is collected for all namespaces. VM=<vm_name>-
Collect details about a particular virtual machine. To use this option, you must also specify a namespace by using the
NSenvironment variable. PROS=<number_of_processes>Modify the maximum number of parallel processes that the
tool uses. The default value ismust-gather.5ImportantUsing too many parallel processes can cause performance issues. Increasing the maximum number of parallel processes is not recommended.
Scripts
Each script is only compatible with certain environment variable combinations.
gather_vms_details-
Collect VM log files, VM definitions, and namespaces (and their child objects) that belong to OpenShift Virtualization resources. If you use this parameter without specifying a namespace or VM, the
must-gathertool collects this data for all VMs in the cluster. This script is compatible with all environment variables, but you must specify a namespace if you use theVMvariable. gather-
Use the default
must-gatherscript, which collects cluster data from all namespaces and includes only basic VM information. This script is only compatible with thePROSvariable. gather_images-
Collect image and image stream custom resource information. This script is only compatible with the
PROSvariable.
13.9.3.1.2. Usage and examples Copiar enlaceEnlace copiado en el portapapeles!
Environment variables are optional. You can run a script by itself or with one or more compatible environment variables.
| Script | Compatible environment variable |
|---|---|
|
|
|
|
|
|
|
|
|
To customize the data that
must-gather
--
Syntax
$ oc adm must-gather \
--image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8:v4.8.7 \
-- <environment_variable_1> <environment_variable_2> <script_name>
Detailed VM information
The following command collects detailed VM information for the
my-vm
mynamespace
$ oc adm must-gather \
--image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8:v4.8.7 \
-- NS=mynamespace VM=my-vm gather_vms_details
- 1
- The
NSenvironment variable is mandatory if you use theVMenvironment variable.
Default data collection limited to three parallel processes
The following command collects default
must-gather
$ oc adm must-gather \
--image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8:v4.8.7 \
-- PROS=3 gather
Image and image stream information
The following command collects image and image stream information from the cluster:
$ oc adm must-gather \
--image=registry.redhat.io/container-native-virtualization/cnv-must-gather-rhel8:v4.8.7 \
-- gather_images
Legal Notice
Copiar enlaceEnlace copiado en el portapapeles!
Copyright © Red Hat
OpenShift documentation is licensed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).
Modified versions must remove all Red Hat trademarks.
Portions adapted from https://github.com/kubernetes-incubator/service-catalog/ with modifications by Red Hat.
Red Hat, Red Hat Enterprise Linux, the Red Hat logo, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of the OpenJS Foundation.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.