Este conteúdo não está disponível no idioma selecionado.
Chapter 20. Diagnosing virtual machine problems
When working with virtual machines (VMs), you might encounter problems with varying levels of complexity and severity. For more complex problems, you might have to capture VM-related data and logs to report or diagnose the problems.
The following sections provide detailed information about generating logs and diagnosing some common VM problems, as well as about reporting these problems.
20.1. Generating libvirt debug logs Copiar o linkLink copiado para a área de transferência!
To diagnose virtual machine (VM) problems, it is helpful to generate and review libvirt debug logs. Attaching debug logs is also useful when asking for support to resolve VM-related problems.
20.1.1. Understanding libvirt debug logs Copiar o linkLink copiado para a área de transferência!
Debug logs are text files that contain data about events that occur during virtual machine (VM) runtime. The logs provide information about fundamental server-side functionalities, such as host libraries and the libvirt daemon. The log files also contain the standard error output (stderr) of all running VMs.
Debug logging is not enabled by default and has to be enabled when libvirt starts.
-
To collect
libvirtdebug logs for your current session, see Enabling libvirt debug logs during runtime. -
To collect
libvirtdebug logs by default, see Enabling libvirt debug logs persistently.
Afterwards, you can attach the logs when requesting support with a VM problem. For details, see Attaching libvirt debug logs to support requests.
20.1.2. Enabling libvirt debug logs persistently Copiar o linkLink copiado para a área de transferência!
To ensure that your virtualization host automatically logs debug information, configure libvirt debug logging to be automatically enabled whenever libvirt starts.
By default, virtqemud is the main libvirt daemon in RHEL 10. To make persistent changes in the libvirt configuration, you must edit the virtqemud.conf file, located in the /etc/libvirt directory.
Procedure
-
Open the
virtqemud.conffile in an editor. Replace or set the filters according to your requirements.
Expand Table 20.1. Debugging filter values 1
logs all messages generated by
libvirt.2
logs all non-debugging information.
3
logs all warning and error messages. This is the default value.
4
logs only error messages.
Sample daemon settings for logging filters
The following settings:
-
Log all error and warning messages from the
remote,util.json, andrpclayers -
Log only error messages from the
eventlayer. -
Save the filtered logs to
/var/log/libvirt/libvirt.log
log_filters="3:remote 4:event 3:util.json 3:rpc" log_outputs="1:file:/var/log/libvirt/libvirt.log"-
Log all error and warning messages from the
- Save and exit.
Restart the
libvirtdaemon.$ systemctl restart virtqemud.service
20.1.3. Enabling libvirt debug logs during runtime Copiar o linkLink copiado para a área de transferência!
To temporarily gather virtualization debugging information, you can modify the libvirt daemon’s runtime settings to enable debug logs and save them to an output file.
This is useful when restarting the libvirt daemon is not possible because restarting fixes the problem, or because there is another process, such as migration or backup, running at the same time. Modifying runtime settings is also useful if you want to try a command without editing the configuration files or restarting the daemon.
Prerequisites
-
Make sure the
libvirt-adminpackage is installed.
Procedure
Optional: Back up the active set of log filters.
# virt-admin -c virtqemud:///system daemon-log-filters >> virt-filters-backupThis makes it possible to restore the active set of filter after generating the logs. If you do not restore the filters, the messages continue to be logged, which may affect system performance.
Use the
virt-adminutility to enable debugging and set the filters according to your requirements.Expand Table 20.2. Debugging filter values 1
logs all messages generated by libvirt.
2
logs all non-debugging information.
3
logs all warning and error messages. This is the default value.
4
logs only error messages.
Sample virt-admin setting for logging filters
The following command:
-
Logs all error and warning messages from the
remote,util.json, andrpclayers -
Logs only error messages from the
eventlayer.
# virt-admin -c virtqemud:///system daemon-log-filters "3:remote 4:event 3:util.json 3:rpc"-
Logs all error and warning messages from the
Use the
virt-adminutility to save the logs to a specific file or directory.For example, the following command saves the log output to the
libvirt.logfile in the/var/log/libvirt/directory.# virt-admin -c virtqemud:///system daemon-log-outputs "1:file:/var/log/libvirt/libvirt.log"Optional: You can also remove the filters to generate a log file that contains all VM-related information. However, it is not recommended since this file may contain a large amount of redundant information produced by libvirt’s modules.
Use the
virt-adminutility to specify an empty set of filters.# virt-admin -c virtqemud:///system daemon-log-filters Logging filters:
Optional: Restore the filters to their original state by using the backup file that you created previously.
# virt-admin -c virtqemud:///system daemon-log-filters "<original-filters>"In this command, replace
<original-filters>with the content ofvirt-filters-backup.Note that if you do not restore the filters, the messages continue to be logged, which may affect system performance.
20.1.4. Attaching libvirt debug logs to support requests Copiar o linkLink copiado para a área de transferência!
To diagnose and resolve virtual machine (VM) problems, you might have to request additional support. Attaching the debug logs to the support request is highly recommended to ensure that the support team has access to all the information they need to provide a quick resolution of the VM-related problem.
Procedure
- To report a problem and request support, open a support case.
Based on the encountered problems, attach the following logs along with your report:
-
For problems with the
libvirtservice, attach the/var/log/libvirt/libvirt.logfile from the host. For problems with a specific VM, attach its respective log file.
For example, for the testguest1 VM, attach the
testguest1.logfile, which can be found at/var/log/libvirt/qemu/testguest1.log.
-
For problems with the
20.2. Dumping a virtual machine core Copiar o linkLink copiado para a área de transferência!
To analyze why a virtual machine (VM) crashed or malfunctioned, you can dump the VM core to a file on disk for later analysis and diagnostics.
20.2.1. How virtual machine core dumping works Copiar o linkLink copiado para a área de transferência!
Virtual machine (VM) core dumps save information about the state of a VM to help you debug it.
A VM requires numerous running processes to work accurately and efficiently. In some cases, a running VM might terminate unexpectedly or malfunction while you are using it. Restarting the VM might cause the data to be reset or lost, which makes it difficult to diagnose the exact problem that caused the VM to crash.
In such cases, you can use the virsh dump utility to save (or dump) the core of a VM to a file before you reboot the VM. The core dump file contains a raw physical memory image of the VM which contains detailed information about the VM. This information can be used to diagnose VM problems, either manually, or by using a tool such as the crash utility.
20.2.2. Creating a virtual machine core dump file Copiar o linkLink copiado para a área de transferência!
A virtual machine (VM) core dump contains detailed information about the state of a VM at any given time. This information is similar to a snapshot of the VM, and can help you detect problems if a VM malfunctions or shuts down suddenly.
Prerequisites
- Make sure you have sufficient disk space to save the file. Note that the space occupied by the VM core dump depends on the amount of RAM allocated to the VM.
Procedure
Use the
virsh dumputility.For example, the following command dumps the
test-guest1VM’s cores, its memory and the CPU common register file tosample-core.filein the/core/filedirectory.# virsh dump test-guest1 /core/file/sample-core.file --memory-only Domain 'test-guest1' dumped to /core/file/sample-core.fileImportantThe
crashutility no longer supports the default file format of thevirsh dumpcommand. To analyze a core dump file by usingcrash, you must create the file with the--memory-onlyoption.Additionally, you must use the
--memory-onlyoption when creating a core dump file to attach to a Red Hat Support Case.
Troubleshooting
If the virsh dump command fails with a System is deadlocked on memory error, ensure you are assigning sufficient memory for the core dump file. To do so, use the following crashkernel option value. Alternatively, do not use crashkernel at all, which assigns core dump memory automatically.
crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M
20.3. Backtracing virtual machine processes Copiar o linkLink copiado para a área de transferência!
When a process related to a virtual machine (VM) malfunctions, you can use the gstack command along with the process identifier (PID) to generate an execution stack trace of the malfunctioning process. If the process is a part of a thread group, then all the threads are traced as well.
Prerequisites
Ensure that the
GDBpackage is installed.For details about installing
GDBand the available components, see Installing the GNU Debugger.Make sure you know the PID of the processes that you want to backtrace.
You can find the PID by using the
pgrepcommand followed by the name of the process. For example:# pgrep libvirt 22014 22025
Procedure
Use the
gstackutility followed by the PID of the process you want to backtrace.For example, the following command backtraces the
libvirtprocess with the PID 22014.# gstack 22014 Thread 3 (Thread 0x7f33edaf7700 (LWP 22017)): #0 0x00007f33f81aef21 in poll () from /lib64/libc.so.6 #1 0x00007f33f89059b6 in g_main_context_iterate.isra () from /lib64/libglib-2.0.so.0 #2 0x00007f33f8905d72 in g_main_loop_run () from /lib64/libglib-2.0.so.0 ...For more information on using
gstack, see thegstackman page on your system.
20.4. Additional resources for reporting virtual machine problems and providing logs Copiar o linkLink copiado para a área de transferência!
To request additional help and support with virtual machines, as well as other Red Hat software, you can do any of the following:
Raise a service request by using the redhat-support-tool command line option, the Red Hat Portal UI, or several methods of FTP.
- To report problems and request support, see Open a Support Case.
Upload the SOS Report and the log files when you submit a service request.
This ensures that the Red Hat support engineer has all the necessary diagnostic information for reference.
- For more information about SOS reports, see the Red Hat Knowledgebase solution What is an SOS Report and how to create one in Red Hat Enterprise Linux?
- For information about attaching log files, see the Red Hat Knowledgebase solution How to provide files to Red Hat Support?