Chapter 15. Attaching host devices to virtual machines
You can expand the functionality of a virtual machine (VM) by attaching a host device to the VM. When attaching a host device to the VM, a virtual device is used for this purpose, which is a software abstraction of the hardware device.
15.1. How virtual devices work
Just like physical machines, virtual machines (VMs) require specialized devices to provide functions to the system, such as processing power, memory, storage, networking, or graphics. Physical systems usually use hardware devices for these purposes. However, because VMs work as software processes, they need to use software abstractions of such devices instead, referred to as virtual devices.
The basics of virtual devices
Virtual devices attached to a VM can be configured when creating the VM, and can also be managed on an existing VM. Generally, virtual devices can be attached or detached from a VM only when the VM is shut off, but some can be added or removed when the VM is running. This feature is referred to as device hot plug and hot unplug.
When creating a new VM, libvirt
automatically creates and configures a default set of essential virtual devices, unless specified otherwise by the user. These are based on the host system architecture and machine type, and usually include:
- the CPU
- memory
- a keyboard
- a network interface controller (NIC)
- various device controllers
- a video card
- a sound card
To manage virtual devices after the VM is created, use the command line (CLI) or the RHEL 10 web console.
Performance or flexibility
For some types of devices, RHEL 10 supports multiple implementations, often with a trade-off between performance and flexibility.
For example, the physical storage used for virtual disks can be represented by files in various formats, such as qcow2
or raw
, and presented to the VM by using a variety of controllers:
- an emulated controller
-
virtio-scsi
-
virtio-blk
An emulated controller is slower than a virtio
controller, because virtio
devices are designed specifically for virtualization purposes. However, emulated controllers make it possible to run operating systems that have no drivers for virtio
devices. Similarly, virtio-scsi
offers a more complete support for SCSI commands, and makes it possible to attach a larger number of disks to the VM. Finally, virtio-blk
provides better performance than both virtio-scsi
and emulated controllers, but a more limited range of use cases. For example, attaching a physical disk as a LUN device to a VM is not possible when using virtio-blk
.
For more information about types of virtual devices, see Types of virtual devices.
15.2. Types of virtual devices
Virtualization in RHEL 10 can present several distinct types of virtual devices that you can attach to virtual machines (VMs):
- Emulated devices
Emulated devices are software implementations of widely used physical devices. Drivers designed for physical devices are also compatible with emulated devices. Therefore, emulated devices can be used very flexibly.
However, because they need to faithfully emulate a particular type of hardware, emulated devices might suffer a significant performance loss compared with the corresponding physical devices or more optimized virtual devices.
The following types of emulated devices are supported:
- Virtual CPUs (vCPUs), with a large choice of CPU models available. The performance impact of emulation depends significantly on the differences between the host CPU and the emulated vCPU.
- Emulated system components, such as PCI bus controllers.
- Emulated storage controllers, such as SATA, SCSI or even IDE.
- Emulated sound devices, such as ICH9, ICH6 or AC97.
- Emulated graphics cards, such as VGA cards.
- Emulated network devices, such as rtl8139.
- Paravirtualized devices
Paravirtualization provides a fast and efficient method for exposing virtual devices to VMs. Paravirtualized devices expose interfaces that are designed specifically for use in VMs, and thus significantly increase device performance. RHEL 10 provides paravirtualized devices to VMs by using the virtio API as a layer between the hypervisor and the VM. The drawback of this approach is that it requires a specific device driver in the guest operating system.
It is recommended to use paravirtualized devices instead of emulated devices for VM whenever possible, notably if they are running I/O intensive applications. Paravirtualized devices decrease I/O latency and increase I/O throughput, in some cases bringing them very close to bare metal performance. Other paravirtualized devices also add functionality to VMs that is not otherwise available.
The following types of paravirtualized devices are supported:
-
The paravirtualized network device (
virtio-net
). Paravirtualized storage controllers:
-
virtio-blk
- provides block device emulation. -
virtio-scsi
- provides more complete SCSI emulation.
-
- The paravirtualized clock.
-
The paravirtualized serial device (
virtio-serial
). -
The balloon device (
virtio-balloon
), used to dynamically distribute memory between a VM and its host. -
The paravirtualized random number generator (
virtio-rng
).
-
The paravirtualized network device (
- Physically shared devices
Certain hardware platforms enable VMs to directly access various hardware devices and components. This process is known as device assignment or passthrough.
When attached in this way, some aspects of the physical device are directly available to the VM as they would be to a physical machine. This provides superior performance for the device when used in the VM. However, devices physically attached to a VM become unavailable to the host, and also cannot be migrated.
Nevertheless, some devices can be shared across multiple VMs. For example, in certain cases a single physical device can provide multiple mediated devices, which can then be assigned to distinct VMs.
The following types of passthrough devices are supported:
- USB, PCI, and SCSI passthrough - expose common industry standard buses directly to VMs to make their specific features available to guest software.
- Single-root I/O virtualization (SR-IOV) - a specification that enables hardware-enforced isolation of PCI Express resources. This makes it safe and efficient to partition a single physical PCI resource into virtual PCI functions. It is commonly used for network interface cards (NICs).
- N_Port ID virtualization (NPIV) - a Fibre Channel technology to share a single physical host bus adapter (HBA) with multiple virtual ports.
- GPUs and vGPUs - accelerators for specific kinds of graphic or compute workloads. Some GPUs can be attached directly to a VM, while certain types also offer the ability to create virtual GPUs (vGPUs) that share the underlying physical hardware.
Some devices of these types might be unsupported or not compatible with RHEL. If you require assistance with setting up virtual devices, consult Red Hat support.
15.3. Attaching USB devices to virtual machines by using the command line
When using a virtual machine (VM), you can access and control a USB device, such as a flash drive or a web camera, that is attached to the host system. In this scenario, the host system passes control of the device to the VM. This is also known as a USB-passthrough.
To attach a USB device to a VM, you can include the USB device information in the XML configuration file of the VM.
Prerequisites
- Ensure the device you want to pass through to the VM is attached to the host.
Procedure
Locate the bus and device values of the USB that you want to attach to the VM.
For example, the following command displays a list of USB devices attached to the host. The device we will use in this example is attached on bus 001 as device 005.
lsusb
# lsusb [...] Bus 001 Device 003: ID 2567:0a2b Intel Corp. Bus 001 Device 005: ID 0407:6252 Kingston River 2.0 [...]
Copy to Clipboard Copied! Use the
virt-xml
utility along with the--add-device
argument.For example, the following command attaches a USB flash drive to the
example-VM-1
VM.virt-xml example-VM-1 --add-device --hostdev 001.005
# virt-xml example-VM-1 --add-device --hostdev 001.005 Domain 'example-VM-1' defined successfully.
Copy to Clipboard Copied! NoteTo attach a USB device to a running VM, add the
--update
argument to the command.
Verification
Use the
virsh dumpxml
command to see if the device’s XML definition has been added to the <devices> section in the VM’s XML configuration file.virsh dumpxml example-VM-1
# virsh dumpxml example-VM-1 [...] <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x0407'/> <product id='0x6252'/> <address bus='1' device='5'/> </source> <alias name='hostdev0'/> <address type='usb' bus='0' port='3'/> </hostdev> [...]
Copy to Clipboard Copied! - Run the VM and test if the device is present and works as expected.
15.4. Attaching PCI devices to virtual machines by using the command line
When using a virtual machine (VM), you can access and control a PCI device, such as a storage or network controller, that is attached to the host system. In this scenario, the host system passes control of the device to the VM. This is also known as a PCI device assignment, or PCI passthrough.
To use a PCI hardware device attached to your host in a virtual machine (VM), you can detach the device from the host and assign it to the VM.
Prerequisites
If your host is using the IBM Z architecture, the
vfio
kernel modules must be loaded on the host. To verify, use the following command:lsmod | grep vfio
# lsmod | grep vfio
Copy to Clipboard Copied! The output must contain the following modules:
-
vfio_pci
-
vfio_pci_core
-
vfio_iommu_type1
-
Procedure
The following steps describe generic PCI device assignment. For instructions on assigning specific types of PCI devices, see the following procedures:
- Attaching SR-IOV network devices to virtual machines
- Assigning a GPU to a virtual machine
Obtain the PCI address identifier of the device that you want to use. For example, if you want to use a NVME disk attached to the host, the following output showss it as device
0000:65:00.0
.lspci -nkD
# lspci -nkD 0000:00:00.0 0600: 8086:a708 (rev 01) Subsystem: 17aa:230e Kernel driver in use: igen6_edac Kernel modules: igen6_edac 0000:00:02.0 0300: 8086:a7a1 (rev 04) Subsystem: 17aa:230e Kernel driver in use: i915 Kernel modules: i915, xe 0000:00:04.0 1180: 8086:a71d (rev 01) Subsystem: 17aa:230e Kernel driver in use: thermal_pci Kernel modules: processor_thermal_device_pci 0000:00:05.0 0604: 8086:a74d (rev 01) Subsystem: 17aa:230e Kernel driver in use: pcieport 0000:00:07.0 0604: 8086:a76e (rev 01) Subsystem: 17aa:230e Kernel driver in use: pcieport 0000:65:00.0 0108: 144d:a822 (rev 01) DeviceName: PCIe SSD in Slot 0 Bay 2 Subsystem: 1028:1fd9 Kernel driver in use: nvme Kernel modules: nvme 0000:6a:00.0 0108: 1179:0110 (rev 01) DeviceName: PCIe SSD in Slot 11 Bay 2 Subsystem: 1028:1ffb Kernel driver in use: nvme Kernel modules: nvme
Copy to Clipboard Copied! Open the XML configuration of the VM to which you want to attach the PCI device.
virsh edit vm-name
# virsh edit vm-name
Copy to Clipboard Copied! Add the following
<hostdev>
configuration to the<devices>
section of the XML file.Replace the values on the
address
line with the PCI address of your device. Optionally, to change the PCI address that the device will use in the VM, you can configure a different address on the<address type="pci">
line.For example, if the device address on the host is
0000:65:00.0
, and you want it to use0000:02:00.0
in the guest, use the following configuration:<hostdev mode="subsystem" type="pci" managed="yes"> <driver name="vfio"/> <source> <address domain="0x0000" bus="0x65" slot="0x00" function="0x0"/> </source> <address type="pci" domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </hostdev>
<hostdev mode="subsystem" type="pci" managed="yes"> <driver name="vfio"/> <source> <address domain="0x0000" bus="0x65" slot="0x00" function="0x0"/> </source> <address type="pci" domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </hostdev>
Copy to Clipboard Copied! Optional: On IBM Z hosts, you can modify how the guest operating system will detect the PCI device. To do this, add a
<zpci>
sub-element to the<address>
element. In the<zpci>
line, you can adjust theuid
andfid
values, which modifies the PCI address and function ID of the device in the guest operating system.<hostdev mode="subsystem" type="pci" managed="yes"> <driver name="vfio"/> <source> <address domain="0x0000" bus="0x65" slot="0x00" function="0x0"/> </source> <address type="pci" domain='0x0000' bus='0x02' slot='0x00' function='0x0'> <zpci uid="0x0008" fid="0x001807"/> </address> </hostdev>
<hostdev mode="subsystem" type="pci" managed="yes"> <driver name="vfio"/> <source> <address domain="0x0000" bus="0x65" slot="0x00" function="0x0"/> </source> <address type="pci" domain='0x0000' bus='0x02' slot='0x00' function='0x0'> <zpci uid="0x0008" fid="0x001807"/> </address> </hostdev>
Copy to Clipboard Copied! In this example:
-
uid="0x0008"
sets the domain PCI address of the device in the VM to0008:00:00.0
. fid="0x001807"
sets the slot value of the device to0x001807
. As a result, the device configuration in the file system of the VM is saved to/sys/bus/pci/slots/00001087/address
.If these values are not specified,
libvirt
configures them automatically.
-
- Save the XML configuration.
If the VM is running, shut it down.
virsh shutdown vm-name
# virsh shutdown vm-name
Copy to Clipboard Copied!
Verification
- Start the VM and log in to its guest operating system.
In the guest operating system, confirm that the PCI device is listed.
For example, if you configured guest device address as
0000:02:00.0
, use the following command:lspci -nkD | grep 0000:02:00.0
# lspci -nkD | grep 0000:02:00.0 0000:02:00.0 8086:9a09 (rev 01)
Copy to Clipboard Copied!
15.5. Attaching host devices to virtual machines by using the web console
To add specific functionalities to your virtual machine (VM), you can use the web console to attach host devices to the VM.
Prerequisites
You have installed the RHEL 10 web console.
For instructions, see Installing and enabling the web console.
If you are attaching PCI devices, ensure that the status of the
managed
attribute of thehostdev
element is set toyes
.NoteWhen attaching PCI devices to your VM, do not omit the
managed
attribute of thehostdev
element, or set it tono
. If you do so, PCI devices cannot automatically detach from the host when you pass them to the VM. They also cannot automatically reattach to the host when you turn off the VM.As a consequence, the host might become unresponsive or shut down unexpectedly.
You can find the status of the
managed
attribute in your VM’s XML configuration. The following example opens the XML configuration of theexample-VM-1
VM.virsh edit example-VM-1
# virsh edit example-VM-1
Copy to Clipboard Copied! - Back up important data from the VM.
Optional: Back up the XML configuration of your VM. For example, to back up the
example-VM-1
VM:virsh dumpxml example-VM-1 > example-VM-1.xml
# virsh dumpxml example-VM-1 > example-VM-1.xml
Copy to Clipboard Copied! - The web console VM plug-in is installed on your system.
Procedure
Log in to the RHEL 10 web console.
For details, see Logging in to the web console.
In the
interface, click the VM to which you want to attach a host device.A new page opens with an Overview section with basic information about the selected VM and a Console section to access the VM’s graphical interface.
Scroll to
.The Host devices section displays information about the devices attached to the VM and options to Add or Remove devices.
Click
.The Add host device dialog is displayed.
- Select the device you want to attach to the VM.
Click
The selected device is attached to the VM.
Verification
- Run the VM and check if the device is displayed in the Host devices section.
15.6. Removing USB devices from virtual machines by using the command line
To remove a USB device from a virtual machine (VM), you can remove the USB device information from the XML configuration of the VM.
Procedure
Locate the bus and device values of the USB that you want to remove from the VM.
For example, the following command displays a list of USB devices attached to the host. The device we will use in this example is attached on bus 001 as device 005.
lsusb
# lsusb [...] Bus 001 Device 003: ID 2567:0a2b Intel Corp. Bus 001 Device 005: ID 0407:6252 Kingston River 2.0 [...]
Copy to Clipboard Copied! Use the
virt-xml
utility along with the--remove-device
argument.For example, the following command removes a USB flash drive, attached to the host as device 005 on bus 001, from the
example-VM-1
VM.virt-xml example-VM-1 --remove-device --hostdev 001.005
# virt-xml example-VM-1 --remove-device --hostdev 001.005 Domain 'example-VM-1' defined successfully.
Copy to Clipboard Copied!
To remove a USB device from a running VM, add the --update
argument to the previous command.
Verification
- Run the VM and check if the device has been removed from the list of devices.
15.7. Removing PCI devices from virtual machines by using the command line
To remove a PCI device from a virtual machine (VM), remove the device information from the XML configuration of the VM.
Procedure
In the XML configuration of the VM to which the PCI device is attached, locate the
<address domain>
line in the<hostdev>
section with the device’s setting.virsh dumpxml <VM-name>
# virsh dumpxml <VM-name> [...] <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x65' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </hostdev> [...]
Copy to Clipboard Copied! Use the
virsh detach-device
command with the the--hostdev
option and the device address.For example, the following command persistently removes the device located in the previous step.
virt detach-device <VM-name> --hostdev 0000:65:00.0 --config
# virt detach-device <VM-name> --hostdev 0000:65:00.0 --config Domain 'VM-name' defined successfully.
Copy to Clipboard Copied! NoteTo remove a PCI device from a running VM, add the
--live
argument to the previous command.Optional: Re-attach the PCI device to the host. For example the following command re-attaches the device removed from the VM in the previous step:
virsh nodedev-reattach pci_0000_65_00_0
# virsh nodedev-reattach pci_0000_65_00_0 Device pci_0000_65_00_0 re-attached
Copy to Clipboard Copied!
Verification
Display the XML configuration of the VM again, and check that the
<hostdev>
section of the device no longer appears.virsh dumpxml <VM-name>
# virsh dumpxml <VM-name>
Copy to Clipboard Copied!
15.8. Removing host devices from virtual machines by using the web console
To free up resources, modify the functionalities of your VM, or both, you can use the web console to modify the VM and remove host devices that are no longer required.
Prerequisites
You have installed the RHEL 10 web console.
For instructions, see Installing and enabling the web console.
- The web console VM plug-in is installed on your system.
Optional: Back up the XML configuration of your VM by using
virsh dumpxml example-VM-1
and sending the output to a file. For example, the following backs up the configuration of your testguest1 VM as thetestguest1.xml
file:virsh dumpxml testguest1 > testguest1.xml cat testguest1.xml
# virsh dumpxml testguest1 > testguest1.xml # cat testguest1.xml <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>testguest1</name> <uuid>ede29304-fe0c-4ca4-abcd-d246481acd18</uuid> [...] </domain>
Copy to Clipboard Copied!
Procedure
In the
interface, click the VM from which you want to remove a host device.A new page opens with an Overview section with basic information about the selected VM and a Console section to access the VM’s graphical interface.
Scroll to
.The Host devices section displays information about the devices attached to the VM and options to Add or Remove devices.
Click the
button next to the device you want to remove from the VM.A remove device confirmation dialog is displayed.
Click
.The device is removed from the VM.
Troubleshooting
If removing a host device causes your VM to become unbootable, use the
virsh define
utility to restore the XML configuration by reloading the XML configuration file you backed up previously.virsh define testguest1.xml
# virsh define testguest1.xml
Copy to Clipboard Copied!
15.9. Attaching ISO images to virtual machines
When using a virtual machine (VM), you can access information stored in an ISO image on the host. To do so, attach the ISO image to the VM as a virtual optical drive, such as a CD drive or a DVD drive.
15.9.1. Attaching ISO images to virtual machines by using the command line
To attach an ISO image as a virtual optical drive, edit the XML configuration file of the virtual machine (VM) and add the new drive.
Prerequisites
- You must store and copy path of the ISO image on the host machine.
Procedure
Use the
virt-xml
utility with the--add-device
argument:For example, the following command attaches the
example-ISO-name
ISO image, stored in the/home/username/Downloads
directory, to theexample-VM-name
VM.virt-xml example-VM-name --add-device --disk /home/username/Downloads/example-ISO-name.iso,device=cdrom
# virt-xml example-VM-name --add-device --disk /home/username/Downloads/example-ISO-name.iso,device=cdrom Domain 'example-VM-name' defined successfully.
Copy to Clipboard Copied!
Verification
- Run the VM and test if the device is present and works as expected.
15.9.2. Replacing ISO images in virtual optical drives
To replace an ISO image attached as a virtual optical drive to a virtual machine (VM), edit the XML configuration file of the VM and specify the replacement.
Prerequisites
- You must store the ISO image on the host machine.
- You must know the path to the ISO image.
Procedure
Locate the target device where the ISO image is attached to the VM. You can find this information in the VM’s XML configuration file.
For example, the following command displays the
example-VM-name
VM’s XML configuration file, where the target device for the virtual optical drive issda
.virsh dumpxml example-VM-name
# virsh dumpxml example-VM-name ... <disk> ... <source file='$(/home/username/Downloads/example-ISO-name.iso)'/> <target dev='sda' bus='sata'/> ... </disk> ...
Copy to Clipboard Copied! Use the
virt-xml
utility with the--edit
argument.For example, the following command replaces the
example-ISO-name
ISO image, attached to theexample-VM-name
VM at targetsda
, with theexample-ISO-name-2
ISO image stored in the/dev/cdrom
directory.virt-xml example-VM-name --edit target=sda --disk /dev/cdrom/example-ISO-name-2.iso
# virt-xml example-VM-name --edit target=sda --disk /dev/cdrom/example-ISO-name-2.iso Domain 'example-VM-name' defined successfully.
Copy to Clipboard Copied!
Verification
- Run the VM and test if the device is replaced and works as expected.
15.9.3. Removing ISO images from virtual machines by using the command line
To remove an ISO image attached to a virtual machine (VM), edit the XML configuration file of the VM.
Procedure
Locate the target device where the ISO image is attached to the VM. You can find this information in the VM’s XML configuration file.
For example, the following command displays the
example-VM-name
VM’s XML configuration file, where the target device for the virtual optical drive issda
.virsh dumpxml example-VM-name
# virsh dumpxml example-VM-name ... <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sda' bus='sata'/> ... </disk> ...
Copy to Clipboard Copied! Use the
virt-xml
utility with the--remove-device
argument.For example, the following command removes the optical drive attached as target
sda
from theexample-VM-name
VM.virt-xml example-VM-name --remove-device --disk target=sda
# virt-xml example-VM-name --remove-device --disk target=sda Domain 'example-VM-name' defined successfully.
Copy to Clipboard Copied!
Verification
- Confirm that the device is no longer listed in the XML configuration file of the VM.
15.10. Attaching a watchdog device to a virtual machine by using the web console
To force the virtual machine (VM) to perform a specified action when it stops responding, you can attach virtual watchdog devices to a VM.
Prerequisites
You have installed the RHEL 10 web console.
For instructions, see Installing and enabling the web console.
- You have installed the web console VM plugin on your system. For more information, see Section 2.3, “Setting up the web console to manage virtual machines”.
Procedure
On the command line, install the watchdog service.
# dnf install watchdog
- Shut down the VM.
Add the watchdog service to the VM.
# virt-xml vmname --add-device --watchdog action=reset --update
- Run the VM.
Log in to the RHEL 10 web console.
For details, see Logging in to the web console.
- In the interface of the web console, click on the VM to which you want to add the watchdog device.
Click Watchdog field in the Overview pane.
next to theThe Add watchdog device type dialog is displayed.
- Select the action that you want the watchdog device to perform if the VM stops responding.
- Click .
Verification
- The action you selected is visible next to the Watchdog field in the Overview pane.